Top 7 Search Engine Evaluator Jobs at Appen and Lionbridge

This post may contain affiliate links. We may receive a comission from purchases made through these links


Search engine evaluators are remote-based, independent contractors that provide feedback on the performance of web search engines. It is a beginner-friendly, work from home job that requires no experience other than good analytical and web research skills.

Although search evaluation projects/jobs are designed by search engine companies like Google and Bing, the hiring and management process is conducted by third-party crowdsourcing companies. The two most popular ones are Appen and Lionbridge.

The following is a list of the main search engine evaluator jobs that are available through these companies.

Search Engine Evaluator Jobs at Appen

Project Yukon

Yukon involves providing feedback on a variety of different Google Web Search features, products and services. The exam is composed of three parts and is considered difficult by most applicants.

This is the same job known as Personalized Internet Assessor at Lionbridge.

Project Shasta

Shasta aims to improve the content, user relevance, and geographic accuracy of the service provided by Apple Maps. Shasta guidelines are extensive and this is generally considered a search evaluator job of above-average difficulty.

This is the same job known as Online Map Quality Analyst at Lionbridge.

Project Arrow

Arrow is all about evaluating the performance of Google Ads, particularly in terms of ad quality and relevance to the user’s search query. The exam is divided into seven parts, each of which represents a specific type of task that make up this project.

This is the same job known as Personalized Internet Ads Assessor at Lionbridge.

Search Engine Evaluator Jobs at Lionbridge

Personalized Internet Assessor

A Personalized Internet Assessor provides feedback on the services of Google Web Search, particularly on the quality and relevance of search results. Guidelines for this project are nearly 200 pages long and require intense study and attention to detail.

At Appen, this job is known as Project Yukon.

Online Map Quality Analyst

An Online Map Quality Analyst rate the accuracy and relevance of search results from Apple Maps. The exam, which is broken down into two parts, is considered difficult and candidates are typically given two attempts to pass.

At Appen, this job is known as Project Shasta.

Personalized Internet Ads Assessor

A Personalized Internet Ads Assessor reviews online advertisements from Google Ads, particularly in terms of their quality and relevance attributes for the user’s search query. Candidates need to go through many pages of guidelines and exam parts, which makes the qualification process quite extensive and overwhelming to some.

At Appen, this job is known as Project Arrow.

Web Content Assessor

A Web Content Assessor at Lionbridge assigns relevance ratings for search results returned by Microsoft Bing. Basically, the tasks involve assessing how well the landing page satisfies the user’s expectations and intent for the search. Candidates must complete a number of practice tasks for training before they are allowed to take the actual exam.

What makes these the top search engine evaluator jobs at Appen and Lionbridge?

The reason why these jobs were selected as the main search engine evaluator jobs at these companies are as follows:

  • They are popular projects that have been around for years
  • They are related to search engine companies and services that dominate the internet market
  • They are not so easy to get into, but once you are approved, you can expect a relatively steady flow of work for years (as long as you maintain high quality standards)
  • They are available in many different countries

Frequently Asked Questions about Search Engine Evaluator Jobs

How to apply for search engine evaluator jobs?

At Appen, go to Appen Connect and sign up for the Internet Analyst position. Once your application is approved, go ahead and look for a search engine evaluation project on the dashboard. Please note that Appen jobs are always presented in the form of projects. Therefore, instead of looking for “search engine evaluator” on the projects’ page, you must search for the name of the search engine evaluation project, such as YukonArrow and Shasta, for example. Check out this blog post for more details on working from home with Appen,

At Lionbridge, you must look for search engine evaluation job titles on its careers page. Examples of such jobs are Personalized Internet Assessor (Google Web Search), Online Map Quality Analyst (Apple Maps), and Personalized Internet Ads Assessor (Google Ads). Please note that the title of the job may slightly vary or change from time to time. For example, one of the jobs currently listed is titled [Language] speaking Search Engine Evaluators, where [Language] represents a local language, such as Japanese, Chinese, French, etc. Probably, this job title refers to the Bing Web Search project, which used to be known as Web Content Assessor.

How much do search engine evaluator jobs pay?

The hourly pay rate ranges from $3 to $20, depending on the job, company, and country where the remote agent is located. The expected workload is between 10 and 40 hours per week depending on the rater location and seasonality of tasks. Therefore, you can expect to earn anywhere between $120 and $3,200 per month on average.


How to prepare for the search engine evaluator exam?

For all search engine evaluator jobs, it is important that the applicant take their time to read the project guidelines thoroughly before taking the exam, ideally twice. The guidelines are sent to successful applicants about a week before the qualification exam is scheduled by the company’s management team. You can find more details about the application process here.

For Project Yukon (Appen) / Personalized Internet Assessor (Lionbridge), you can get some extra help by taking the courses that are available at the Search Evaluator Academy. Although we cannot guarantee that they will make you pass the test, many applicants report that these courses make them more comfortable with the nuances of web search evaluation. In turn, that can help them feel more confident during the test.

More details about the Yukon / Internet Assessor exam preparation can be found here.

2 thoughts on “Top 7 Search Engine Evaluator Jobs at Appen and Lionbridge”

  1. Hi
    I purchased full course and prepared pretty decently for Search Evaluator Role at Lionbridge. I cleared theoretical test but couldn’t clear PQ rating test. I am looking for some quick n quality help before I get a second chance. I found your course to be really helpful for needs met test, however I am not able to get enough feedback and improve upon my Page quality test skills. Can u help or can u point to someone/something which can help. Thanks for your help in advance.

Leave a Reply

Your email address will not be published. Required fields are marked *