QA Reviewer / Adjudicator

QA Reviewer / Adjudicator

Applications open

Check annotation and transcription work done by others, as paid work from home. It suits precise people who give clear written feedback and reject work that fails the criteria. Prior annotation, transcription or review experience is a plus, not a requirement.

Remote, United States · Freelance

What the work is for

Your reviews decide which work goes into a delivered dataset. Work must be reviewed before a project accepts it.

The work

• Review sampled batches against the written acceptance criteria. Score them using the project's rating criteria.
• Resolve disagreements between annotators. Record your reason and give clear feedback.
• Measure how often annotators agree, and help calibrate them where they drift apart.
• Report recurring errors and guideline gaps so the instructions improve.

What to expect

• Review and adjudication run in YPAI's own review console, next to the tool the annotators used.
• Each project has its own acceptance criteria, and batches can be annotation or transcription.

You need

• Precision, and the willingness to reject work that fails the written criteria.
• Structured, consistent judgement and clear written feedback.
• 18 years or older.

Good to know

• You judge by the written criteria, even where you would have labelled an item differently.
• Work comes project by project, and there can be gaps between projects.

Our team reviews applications for this role and contacts selected applicants by email.

Apply

Takes a couple of minutes. No CV needed.

Payment

Via PayPal, Wise, or bank transfer

When you are paid

Paid every two weeks after approved work.

Fees to you

Never, at any stage. If someone asks, it is not us

Who hires you

YPAI, a registered Norwegian company that collects and annotates data for AI

Website

Full name

Email

Phone (optional)

CountrySelect...United States──────────AfghanistanÅland IslandsAlbaniaAlgeriaAmerican SamoaAndorraAngolaA

State (optional)Select...AlabamaAlaskaArizonaArkansasCaliforniaColoradoConnecticutDelawareDistrict of ColumbiaFloridaGeorgiaHawaiiIdahoIllinoisIndianaIowaKansasKentuckyLouisianaMaineMarylandMassachusettsMichiganMinnesotaMississippiMissouriMontanaNebraskaNevadaNew HampshireNew JerseyNew MexicoNew YorkNorth CarolinaNorth DakotaOhioOklahomaOregonPennsylvaniaRhode IslandSouth CarolinaSouth DakotaTennesseeTexasUtahVermontVirginiaWashingtonWest VirginiaWisconsinWyoming Add a CVoptional PDF, Word or a photo, max 5 MB. Skip it if you do not have one. Send application

You see the confirmation here.

Application received

Thank you for applying for QA Reviewer / Adjudicator. Our team will review your application and contact you by email if you are selected.

• We have emailed you a receipt.
• Your YPAI Freelance profile is set up too. Use the link in the email to complete, edit or delete it.
• Your YPAI Freelance profile is set up too. Sign in with this email address to complete, edit or delete it.
• This application is saved on your YPAI Freelance profile.

Your application, and any CV you uploaded, are deleted no later than 2 years after you applied.

YPAIlinkedin.com/company/ypai-ai Follow YPAI on LinkedIn

Typical project pay

USD 28-40 per hour

Each project invitation states the work, your pay, what the pay covers and what the work must meet. You see it before you accept.

Where

Remote, United States

Contract

Freelance

Status

Applications open

Apply now Takes a couple of minutes. No CV needed.