
Remote Intelligence Analyst (Strategic Defense & Assessment) — Next-Gen AI Model Evaluator
TheTop Talent Group manages an invitation-only bench of senior knowledge workers deployed on mission-critical cognitive tasks. We have been commissioned to engage an exceptional Remote Intelligence Analyst (Strategic Defense & Assessment) — Next-Gen AI Model Evaluator ($30 - $80/hour) with demonstrated authority in Analytic rigor, Source handling, Over-refusal detection, All-Source Intelligence Analysis. In this capacity, you will author rigorous stress tests, pinpoint logical fallacies, and develop master evaluation frameworks for next-generation frontier intelligence platforms.
Engagement Summary & Primary Objectives
- Identify and flag instances where legitimate analytic exploration is over-refused by the model, providing clear documentation and feedback.
- Document edge cases and analytic disagreements, supporting continuous improvement of evaluation methodologies.
- Collaborate with project leads to refine procedures for source evaluation and confidence-level assignment.
- Contribute insights to help calibrate analytic standards and enhance the overall quality of AI system benchmarks.
- Develop and map analytic cases to defined taxonomies across multiple evaluation areas, ensuring comprehensive coverage of relevant intelligence scenarios.
- Author complex test cases designed to evaluate model performance on source handling, analytic rigor, and nuanced uncertainty expression.
- Assess and grade AI-generated responses based on criteria such as reasoning, source use, recognition of limitations, and practical utility.
Prerequisites & Expert Criteria
- Ability to identify and articulate nuanced model limitations and over-refusal scenarios.
- Familiarity with structured analytic techniques and intelligence tradecraft standards.
- Previous participation in AI evaluation or model assessment initiatives is a plus.
- Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline, ideally with a civilian-IC background.
- Strong proficiency in source evaluation, confidence-level assessment, and established analytic standards.
- Experience in teaching, mentoring, or leading analytic reviews, with a record of fostering analytic rigor.
- Exceptional written and verbal communication skills, with a focus on clarity and precision.
Consulting Compensation & Work Arrangement
- Compensation Rate: $30 - $80/hour (verified contractor remuneration)
- Work Arrangement: 100% Remote / Work From Home
- Eligible Remote Countries: United States, United Kingdom, Canada, Australia, New Zealand, Ireland, and South Africa
- Core Competencies: Analytic rigor, Source handling, Over-refusal detection, All-Source Intelligence Analysis
- Hiring Lead: Julian Montgomery (Managing Director of Specialist Talent) at TheTop Talent Group
Direct Application Process: Click the Apply button on this page to submit your profile directly for immediate evaluation.
Ready to Apply for Remote Intelligence Analyst (Strategic Defense & Assessment) — Next-Gen AI Model Evaluator?
Applications are routed directly to our verified partner application portal for Remote Intelligence Analyst (Strategic Defense & Assessment) — Next-Gen AI Model Evaluator ($30 - $80/hour).
Apply Now for This Position →