LLM Language Evaluation Specialist (Gaeilge)

About the role

We are seeking a Gaeilge language specialist to evaluate AI‑generated content for a long‑term LLM Language Evaluation Project. The role is remote, project‑based and expected to run for at least three months, with an initial paid trial phase.

Key responsibilities

• Assess AI/LLM‑generated Gaeilge text for accuracy, naturalness and linguistic appropriateness following detailed guidelines.
• Identify and classify issues such as mistranslations, missing content, incorrect terminology, spelling, grammar, punctuation, spacing errors, and other readability problems.
• Detect abnormal cases including non‑Gaeilge content, language identification errors, duplicate entries, or content that must be rejected.
• Provide structured evaluation results consistently while maintaining high attention to detail.

Required profile

• High proficiency in Gaeilge.
• Previous experience with LLM evaluation, AI training, data annotation, localization or translation projects.
• Strong understanding of Gaeilge grammar, vocabulary and natural language usage.
• Ability to follow detailed evaluation guidelines and work with digital documents and online tools.

Required skills

• Experience evaluating large language model outputs.
• Familiarity with structured annotation tools.