Job Description
Senior Software Engineer ā Coding Copilot Evaluation
Hourly Contract | Remote | $140/hour
Role Overview
Part-time opportunity to work with a leading AI application for software engineers, focused on evaluating and improving coding copilot systems. This role involves assessing AI-generated engineering solutions, designing structured evaluation rubrics, and contributing to the refinement of next-generation developer tools.
- Key Responsibilities
- Evaluate AI-generated outputs, including proposed architectures, code diffs, code review commentary, and refactoring plans
- Develop rubric-based evaluations to assess code quality, design correctness, resilience, and maintainability
- Collaborate asynchronously with AI and product teams to guide model development and improve evaluation criteria
- Qualifications
- 5+ years of software engineering experience at leading tech companies or high-growth startups
- Strong background in system design and code review processes
- Proven experience creating review assets such as checklists, feedback summaries, and quality trend reports
- Excellent analytical, writing, and presentation skills
- Project Details
- Start Date: Pending confirmation
- Duration: 4 weeks
- Commitment: ~20 hours/week
- Fully remote and asynchronous
- Compensation & Contract
- $140/hour USD
- Independent contractor (W9/1099)
- Paid weekly via Stripe Connect
- Application Process
- Submit resume and application form
- Complete a 30-minute AI interview focused on your technical expertise
- Participate in a rubric calibration assessment (30 minutes) to evaluate AI-generated content
Apply Now
Apply Now