Sama is hiring a
Remote AI Evaluator
\nAbout the Job:\n\nAs an AI Evaluator (both English and Hindi) you will work to help state-of-the-art generative AI models perform better using your language and critical thinking skills. You will be required to ensure quality in a high-investment area for our client’s program to evaluate LLM (Large Language Model) responses to human queries while checking for accuracy and policy sustainability. \n\nThere will be badge tests and an overall training process aimed at verifying the target skills and ensuring that any new candidate has the right aptitude for this work. All activities need to be completed within the defined SLA (service level agreement) and quality. All work will be in both Hindi and English.\n\nThis is a fully remote position, and candidates are required to have their own computer with a reliable high-speed internet connection to perform this role effectively.Key Responsibilities: \n\nKey Responsibilities: \n\n\n* Proficiency in English and Hindi: intermediate or advanced language skills, which the candidate will use to:\n\n\n\n* Analyze and understand the meaning of the model response, the user feedback and how it relates to the response.\n\n* Identify messages that are inappropriate and trigger policy concerns. \n\n* Define the target information (claims) in the response.\n\n* Recognize information that isn’t clearly pointed out as wrong - e.g. missing details that make the response incomplete - or information that’s mixed up between multiple entities (disambiguation). \n\n* Formulate appropriate strings for fact-checking and web research that helps them find the right sources quickly.\n\n* Recognize any differences between the text in the input passage and the text generated in the response, thus identifying input issues.\n\n* Understand the context of the user-model interaction, and recognize when the model doesn’t interpret that context correctly.\n\n* Workflow-related: understand complex Guidelines instructions, ask well-formulated questions, leave informative justifications and task comments.\n\n\n\n* Understand the client platform and follow team level protocols (like how/when to access the task, etc.).\n\n* Strong logic, intuitive problem-solving and critical thinking; adaptability to the needs of the task. The candidate needs to be open-minded and welcome different viewpoints or solutions.\n\n* Proactiveness, confidence to work independently without constant support. Reliability and being able to make good decisions on their own.\n\n* Be a thought partner to delivery managers, input into the project delivery strategies, and share quality best practices to aid in achieving the project goals\n\n\n\n\nMinimum Qualifications:\n\n\n* Have completed a Bachelor's degree.\n\n* Fluency in Hindi at a level equivalent to C2 level.\n\n* Fluency in English at C1 level. \n\n* Digital skills for high levels of research to find accurate information from reliable internet sources.\n\n* Strong critical thinking, reasoning, and exceptional problem-solving skills.\n\n* Excellent attention to detail.\n\n* Experience in GenAI is an added advantage.\n\n* Knowledge of Indian cultural background.\n\n\n\n\nPreferred Qualifications:\n\n\n* Excellent written and oral communication skills in both English and Hindi.\n\n* Hindi qualifications are preferred at Hindi Honors or MPhil levels.\n\n* Demonstrable ability to perform well in a rapidly changing and extremely global team.\n\n* Passion for our mission of ensuring a world-class support experience for our community and customers.\n\n* Experience as an editor is an added advantage.\n\n\n
\n\n#Benefits\n
💰 401(k)\n\n🌎 Distributed team\n\n⏰ Async\n\n🤓 Vision insurance\n\n🦷 Dental insurance\n\n🚑 Medical insurance\n\n🏖 Unlimited vacation\n\n🏖 Paid time off\n\n📆 4 day workweek\n\n💰 401k matching\n\n🏔 Company retreats\n\n🏬 Coworking budget\n\n📚 Learning budget\n\n💪 Free gym membership\n\n🧘 Mental wellness budget\n\n🖥 Home office budget\n\n🥧 Pay in crypto\n\n🥸 Pseudonymous\n\n💰 Profit sharing\n\n💰 Equity compensation\n\n⬜️ No whiteboard interview\n\n👀 No monitoring system\n\n🚫 No politics at work\n\n🎅 We hire old (and young)\n\n
\n\n#Location\nNew Delhi, Delhi, India
Apply for this job
👉 Please reference you found the job on Remote OK, this helps us get more companies to post here, thanks!
When applying for jobs, you should NEVER have to pay to apply. You should also NEVER have to pay to buy equipment which they then pay you back for later. Also never pay for trainings you have to do. Those are scams! NEVER PAY FOR ANYTHING! Posts that link to pages with "how to work online" are also scams. Don't use them or pay for them. Also always verify you're actually talking to the company in the job post and not an imposter. A good idea is to check the domain name for the site/email and see if it's the actual company's main domain name. Scams in remote work are rampant, be careful! Read more to avoid scams. When clicking on the button to apply above, you will leave Remote OK and go to the job application page for that company outside this site. Remote OK accepts no liability or responsibility as a consequence of any reliance upon information on there (external sites) or here.
Frontpage
🏝 Remote jobs
🌗 Dark mode
👩💻 Hire remote workers
🚨 Post a remote job
🍱 Compact mode
✍️ Remote work blog new
Health insurance for teams
Hacker News mode
Safe for work mode
