Member of Technical Staff, Multimodal Vision
HarkSan Jose, California, United States · Posted 5 months agoDescription
About Hark
Hark is an artificial intelligence company building advanced, personalized intelligence. One that is proactive, multimodal, and capable of interacting with the world through speech, text, vision, and persistent memory.
We're pairing that intelligence with next-generation hardware to create a universal interface between humans and machines. While today's AI largely operates through chat boxes and decade-old devices, Hark is focused on what comes next: agentic systems that interact naturally with people and the real world.
To get there, we're developing multimodal models and next-generation AI hardware together - designed from the ground up as a single, unified interface for a new era of intelligent systems.
About the Role
The Omni team at Hark is building the next generation of AI experiences beyond text, enabling models to understand and generate content across multiple modalities, including text, and vision. Our goal is to create seamless, real-time multimodal intelligence that powers intuitive and immersive user experiences.
As part of the Omni team, you will help drive the development of text, video, and multimodal models. This includes working across the full stack—from data and modeling to training, serving, and product integration. You will contribute to both pretraining and posttraining efforts while collaborating closely with product teams to push the boundaries of model capability and deliver exceptional end-to-end user experiences.
Responsibilities
- Drive mid-training and post-training research for multimodal foundation models — continued/mid-training on large-scale interleaved vision-text and video-audio corpora, supervised fine-tuning, and reinforcement learning to advance image, video, and video-audio understanding.
- Own the RL stack for multimodal post-training: reward modeling, preference optimization and RLVR-style verifiable rewards, policy optimization (PPO / GRPO / DPO and variants), rollout and sampling strategies, and mitigation of reward hacking and capability regression.
- Build multimodal evaluation frameworks and internal benchmarks covering perception, reasoning, instruction following, long-video and video-audio understanding, robustness, and RL-specific regressions; turn eval signal into the next training iteration.
- Build and optimize multimodal training infrastructure — distributed training and RL rollout systems, long-context and multi-stream (vision + audio) data loading, throughput and memory efficiency, checkpointing and experiment tooling — for scalability and production deployment.
Requirements
- Proven track record of advancing multimodal or vision-language models through innovations in mid-training, post-training, data, or training systems.
- Hands-on experience with post-training methods at scale — SFT, RLHF/RLAIF, RLVR, reward modeling, and preference optimization — ideally applied to multimodal models.
- Strong experience in image understanding, video modeling, or multimodal learning.
- Strong background in data-driven experimentation, evaluation design, and iterative model development.
Bonus Qualifications
- Experience building or scaling RL training infrastructure.
- Experience with video-audio or speech-language models, long-context video understanding, or streaming and real-time multimodal systems.
- Experience with large-scale machine learning systems and distributed training (e.g., Megatron-LM, FSDP, DeepSpeed, or comparable frameworks).
- Publications at top-tier venues or impactful open-source contributions in multimodal modeling or post-training.
Compensation
The US base salary range for this full-time position is between $180,000 - $450,000 annually.
The pay offered for this position may vary based on several individual factors, including job-related knowledge, skills, and experience. The total compensation package may also include additional components/benefits depending on the specific role. This information will be shared if an employment offer is extended.
Similar jobs
Est. 280,500 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 238,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 280,500 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 242,540 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 280,500 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 280,500 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 238,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 238,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 242,540 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 273,110 USD
Who We Are Nuro is a self-driving technology company on a mission to make autonomy accessible to all. Founded in 2016, Nuro is building the world’s most scalable driver, combining cutting-edge AI with automotive-grade ha…
Est. 229,900 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 390,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 273,110 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 270,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 231,500 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 270,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 193,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 193,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 242,540 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 229,900 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 273,110 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 273,110 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 333,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 238,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 258,875 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 242,540 USD
Who We Are Nuro believes self-driving vehicles are the most immediate and profound opportunity for AI to drive positive change in the physical world. Safer streets, more time for what matters, and easier access to the wo…
Est. 280,500 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 195,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 338,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…
Est. 195,000 USD
Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver—The…