Model Policy, Frontier Cyber Risk | OpenAI Careers ## Model Policy, Frontier Cyber Risk Model Policy - San Francisco Apply now(opens in a new window) **About the Team** Our Safety Systems team is at the forefront of OpenAI's mission to build and deploy safe AGI, driving our commitment to AI safety and fostering a culture of trust and transparency. Within Safety Systems, the Model Policy team aligns model behavior with desired human values and norms. We co-design policy *with* models and *for* models by driving rapid policy taxonomy iteration based on data and defining evaluation criteria for foundational models’ ability to reason about safety. **About the Role** Frontier AI systems are rapidly expanding what is possible in cybersecurity and software engineering. These capabilities create major defensive opportunities, but they also raise serious dual-use and misuse risks across areas such as malware development, exploit discovery, vulnerability chaining, credential abuse, cyber intrusion, and autonomous offensive operations. In this role, you will help define how OpenAI’s models should behave in high-risk cybersecurity contexts. You will develop policy frameworks, threat models, taxonomies, evaluations, and behavioral specifications that guide model behavior across training, deployment, and monitoring systems. This role sits at the intersection of cybersecurity, AI safety, threat modeling, evaluation science, and policy implementation. You will work closely with research, engineering, safety training, preparedness, and product teams to build policies that are technically grounded, measurable, enforceable, and responsive to real-world cyber risk. **Your Responsibilities:** * Design and maintain model policies for cybersecurity and frontier-risk domains, especially dual-use and high-risk cyber capabilities. * Translate cybersecurity threat models into clear behavioral specifications, evaluation criteria, grading guidance, and system-level mitigations. * Define practical boundaries between legitimate security research, defensive workflows, and assistance that could materially enable harmful activity. * Build policy artifacts that support implementation across training, evaluation, deployment, monitoring, and escalation systems. * Partner with safety researchers, engineers, and evaluation teams to operationalize policies into scalable model behavior and measurable safeguards. * Analyze red-teaming results, deployment data, model failures, over-refusals, and ambiguous edge cases to improve policy and evaluation quality over time. * Identify emerging cyber capability areas where advanced AI systems could lower barriers to misuse or increase operational capability for malicious actors. * Contribute to system cards, safety reports, policy documentation, and external communications on OpenAI’s approach to cyber risk mitigation. **We’re Seeking:** * Strong technical expertise in cybersecurity, such as offensive security, defensive security, vulnerability research, malware analysis, incident response, threat intelligence, application security, exploit development, infrastructure security, or cloud security. * Strong judgment about how AI systems may affect the cyber threat landscape, including dual-use, autonomous, or agentic system risks. * Ability to distinguish between legitimate security use cases and assistance that could materially enable harmful cyber activity. * Experience building or applying threat models to complex technical systems, especially in adversarial or high-risk environments. * Ability to translate technical security expertise into structured policy frameworks, evaluation criteria, operational guidance, and enforcement mechanisms. * Comfort using empirical evidence, including evaluations, red-teaming results, deployment observations, and model failure modes, to inform policy decisions. * Strong systems thinking across policy, evaluations, classifiers, training

Model Policy, Frontier Cyber Risk

Classified Tasks (0)

Job description