Mission map · Study map
What to learn, in order
The missions are the journey. This is the map underneath: the concepts and skills each mission rests on, why they matter, and the checkpoint that proves you've got them. Read it to see where you are — and what's still a draft.
The spine of the whole mission map
🦾 arm→
✊ hand→
👁️ eye→
🧠 autonomy
Every mission moves one step along this line. Move the arm, close the hand, add an eye, then hand control to a learned brain (autonomy). The study units below hang off these four.
Honesty note. Physical Spark is built in public and this map says the quiet part out loud: some units are live, some are draft (v0), and Missions 4–6 are still to write. That's the roadmap, not a promise everything exists yet.
Foundation — before the terminal
Concept-first, on your phone. No hardware, no code yet.
GSGround Controllive
You'll learn
Why we train in simulation, the agent–environment loop, Gymnasium's five methods, action spaces & the −1…1 normalization math, and ALOHA teleoperation.
Why
So the hands-on part feels easy, not scary — the vocabulary of robot learning in plain words.
Prereqs
None. A phone and curiosity.
Tools
read-only · MuJoCo / Gymnasium concepts
Checkpoint
You can draw the loop from memory and read Box(-1,1,(14,)) out loud.
The brain — how a robot learns a skill
The learning ideas that sit under Missions 2–3, and power every VLA.
M2Learning by Showing — imitation learninglive
You'll learn
Behavioral cloning; the LeRobotDataset; compounding error (why "just copy" drifts); and the fixes — action chunking (ACT), diffusion policy, and where VLAs fit.
Why
This is the conceptual keystone: Mission 2 records demos and trains a policy, and every 2026 frontier model is built from these pieces.
Tools
LeRobot · ACT · Diffusion Policy · SmolVLA
Checkpoint
You can explain compounding error, and why chunking and diffusion fight it.
×VLA — vision, language, actiondraft (v0)
You'll learn
How a pretrained model turns a camera frame + "pick the red cup" into actions; SmolVLA (runs on a Mac), π0 / GR00T; fine-tuning a VLA on your own demos inside LeRobot.
Why
It's the bridge from "one memorized task" to "follow an instruction on an unseen object" — the target of Mission 6.
Prereqs
Imitation learning (chunking + diffusion).
Tools
SmolVLA · π0 (openpi) · GR00T · LeRobot fine-tuning
Checkpoint
You can fine-tune SmolVLA on a small dataset and describe what changed.
×sim-to-real — crossing the gapto write
You'll learn
Why a policy trained in sim stumbles on real hardware; domain randomization; fine-tuning on a little real data; human-in-the-loop correction.
Why
Every "it worked in sim!" moment meets reality here. It's the difference between a demo and a deployment.
Prereqs
Imitation learning; a real SO-101 (or a hybrid plan).
Tools
MuJoCo / Isaac Lab · domain randomization · HIL
Checkpoint
You can name three reasons a sim policy fails on the real arm and one fix for each.
Eyes & hands — Missions 4 → 6
The largest gap: today these exist only as one line each. This is the plan for filling them.
M4Eyes — depth, point clouds & calibrationto write
You'll learn
- RGB-D depth: how a camera gives distance, not just color.
- Point clouds & object pose estimation.
- Camera calibration (intrinsics) and hand-eye calibration — mapping what the camera sees to where the arm must move.
Why
This is the real inflection point — from "an arm that moves as programmed" to "a robot that sees and decides." No calibration, no reliable grasp.
Prereqs
Ground Control; a webcam, then a RealSense D435i (~$344).
Tools
Intel RealSense D435i · OpenCV · point-cloud basics
Checkpoint
You can turn a pixel the camera sees into an arm target the gripper can reach.
M5Grasp Anything — 6-DoF graspingto write
You'll learn
How to grasp an object the robot has never seen: 6-DoF grasp generation with NVIDIA GraspGen (a diffusion model over grasp poses), scoring and selecting a grasp, then executing it.
Why
Hard-coded grasps break on new objects. Generative grasping is what makes "anything" possible.
Prereqs
Eyes (depth + calibration).
Tools
GraspGen · (AnyGrasp) · parallel-jaw gripper
Checkpoint
The arm grasps a new object placed at a random pose, from perception alone.
M6"pick that up" / "저거 집어" — language → grasp → executeto write
You'll learn
The full 2-stage pipeline: language grounding (LangSAM / Grounded-SAM2 turns "the red cup" into a mask) → GraspGen (a 6-DoF grasp on that object) → motion planning → execute. This IS the final demo — the one that proves the record.
Why
It's the whole point of the arena — the video a recruiter watches. The résumé is a URL, not a PDF.
Prereqs
Eyes + Grasp Anything (+ the VLA unit for context).
Tools
LangSAM / Grounded-SAM2 · GraspGen · LeRobot
Checkpoint
You say "grab that," it picks an unseen object, and the clip passes peer review.
🏆Capstone — the hackathonto write
You'll learn
A shared task with an objective rubric (success rate, speed), a submission format (episode video + dataset/code), and a peer-review scoring flow — the "completion is the product" mechanic, made concrete.
Why
Peer review only works when tied to a reproducible artifact. Robotics gives us one: the policy grabbed the cube, or it didn't.
Tools
the gallery · reproducible submissions
Checkpoint
Your solution is ranked against peers on a number everyone can re-run.
Side track — the ROS2 bridge
Deliberately outside the main line, documented honestly.
↔ROS2 bridge (optional)to write
You'll learn
Enough ROS2 to move from our learning-layer track toward the middleware Korean job postings ask for — nodes, topics, and where a learned policy plugs in.
Why
We prove the policy layer, not middleware — but the market often wants ROS2. This is the documented "next step," to be built the moment a player is blocked on it in hiring.
Checkpoint
You can wrap a trained policy as a ROS2 node.