When Behavior Cloning Cheats

Project Arc

A simple assignment became a controlled robot-learning experiment.

I started with a bin-sorting task: a simulated robot carries a tray, observes a colored cube, and dumps it into the matching bin. The first behavior cloning policy appeared to work. Then I randomized the bins and found the policy had learned a shortcut.

Main lesson A learned robot policy can produce a convincing demo while relying on the wrong signal. In this case, fixed-bin training let the network learn color → memorized joint motion instead of visual scene geometry → target bin location → action.

200 Expert episodes collected

24D State vector in first dataset

50 Hz Approx. recording rate after stride

Δθ Learned joint-delta action representation

Task Setup

Sort the colored cube into the matching bin.

Each episode samples a cube color and asks the robot to deliver it to the matching bin. The current version uses a fixed camera view, robot state, and task information. The long-term motivation is a lightweight VLA-style setup, but the present baseline is deliberately scoped to multimodal behavior cloning.

Observation

Vision + robot state

RGB images from a fixed camera, plus joint state, force/torque, end-effector pose, and object pose.

Task signal

Instruction / color

Simple task strings such as sort the red object into the red bin, plus compact color encoding.

Action

Joint deltas

The expert records absolute joint targets, but the learner predicts normalized joint deltas for stability.

Why this is not yet a full VLA

I began by studying lightweight open-source VLA models such as SmolVLA and LeRobot-style pipelines. For this first implementation, I scoped the problem to behavior cloning with multimodal inputs. The policy consumes images, robot state, and simplified task information, but it is not yet a general language-conditioned vision-language-action model.

Scripted Oracle

First, build an expert that can solve the task by construction.

The oracle policy is a finite-state machine. It reads MuJoCo ground-truth object information, selects the correct bin, moves to a pre-drop pose, tilts the tray to dump the cube, and returns home. This policy is intentionally not learned — its job is to generate demonstrations.

01

Reset episode

Randomize cube color and reset the robot, tray, cube, and bins.

02

Read color

Oracle reads the environment's cube-color attribute directly.

03

Move to pre-drop

IK drives the tray to a hardcoded pose near the selected bin.

04

Tilt tray

The wrist tilts to dump the cube into the matching bin.

05

Record rollout

Images, state, instructions, actions, color, and success labels are saved.

Oracle policy replace source if needed

The oracle uses privileged simulator information, so it should not be confused with the learned policy.

Demo Collection

The dataset is the experiment.

The first dataset recorded 200 successful expert episodes. To keep collection efficient, I saved every Nth physics step rather than every MuJoCo step, and rendered RGB frames only when a sample was actually captured.

Signal	Shape / Example	Purpose
Images	Fixed isometric RGB camera	Visual context for cube and bin geometry.
State	`q, qdot, ft, ee_pos, obj_pos`	Robot proprioception and simulator-derived task state.
Instruction	`sort the red object into the red bin`	Task conditioning, currently simplified rather than full language grounding.
Action	Expert joint targets → converted to `Δθ`	Closed-loop command target for the learned policy.
Cube color	`red` / `blue`	Compact task label and diagnostic variable.
Success	Boolean episode outcome	Tracks whether the matching bin received the cube.

Implementation notes

I initially recorded absolute joint position targets because that is what the expert command produced. During training, I converted these to joint deltas. This made normalization easier and reduced unstable closed-loop behavior during evaluation.

For speed, the recorder uses a stride over physics steps — for example, record_stride=20 yields roughly 50 Hz depending on the simulator step size. RGB rendering is also skipped on steps that are not being recorded.

Behavior Cloning

A compact multimodal policy.

The policy combines image features, normalized robot state, and task embeddings. It predicts a normalized joint-delta action, which is unnormalized and executed in simulation.

RGB image

Fixed camera view of tray, cube, and bins.

→

CNN encoder

Produces image features. Increased spatial output helped avoid over-compressing vision.

→

Concat features
Image features + state + instruction/color embeddings.

Robot state

Normalized q, qdot, force/torque, end-effector pose, object pose.

→

MLP trunk

Learns the mapping from multimodal observation to action.

→

Action head
Predicts normalized joint deltas, trained with MSE against expert actions.

Instruction / color

Task string and/or compact object-color encoding.

→

Embedding

Small learned task representation.

→

Closed-loop eval

Unnormalize action, execute, observe next state, repeat.

Important modeling change Training directly on absolute joint targets produced unstable behavior. Predicting joint deltas gave the learner a smoother, more local control target.

The Interesting Failure

The policy worked — until the bins moved.

With fixed bin positions, the behavior cloning policy appeared to solve the task. It moved roughly like the expert, responded to cube color, and produced plausible trajectories. But when I randomized the bin locations during evaluation, the policy failed.

Shortcut discovered The learned policy was mostly remembering a color-conditioned motion: red goes one way, blue goes the other. It was not sufficiently using the visual location of the bins.

Fixed bins looks good

Initial success

The policy appears to sort correctly when the environment matches the training layout.

Randomized bins fails

Generalization failure

Once bin locations change, the policy exposes that it has not learned the scene geometry.

Dataset Fix

Randomize the expert data, not just the evaluation.

To remove the shortcut, I updated expert data collection so bin locations are randomized during demonstrations. Each bin is placed at a reachable radius and angle around the base, rotated to remain radially consistent, and checked to avoid overlap.

Before

Fixed layout

The policy can solve the training distribution by mapping cube color to a memorized joint-space trajectory.

After

Randomized bins

The expert produces demonstrations across varying reachable bin positions, forcing the policy to attend to scene context.

Randomized bin rollouts expert or learned policy

Suggested clip: either the randomized-bin oracle rollout or the improved policy after retraining on randomized demonstrations.

Randomization details

The randomized-bin version samples each bin on a radius around the robot base and chooses an angle within the reachable workspace. Bins are rotated to be radially symmetric, the pre-drop pose is moved higher in z, and the tray tilts away from the robot along an axis tangent to the circle around the base.

Demo Reel

Four clips tell the whole story.

For the LinkedIn post, I would use only the strongest one. For this page, the failure video is valuable because it shows the diagnostic process, not just the best-looking result.

01oracle

Scripted oracle

Privileged expert policy generates demonstrations.

02initial BC

Initial behavior cloning

Works in the fixed environment, but this is not yet enough evidence.

03failure

Randomized-bin failure

The policy fails when the layout changes, revealing shortcut learning.

04fix

Randomized data collection

The environment now forces visual grounding during demonstration collection.

Next Steps

From behavior cloning baseline toward VLA-style manipulation.

The current project is a behavior cloning baseline with multimodal inputs. The next stage is to increase the visual and language grounding while preserving careful evaluation against shortcut learning.

Perception

Wrist camera

Compare fixed-camera, wrist-camera, and multi-camera policies to study the cost and benefit of additional visual tokens.

Robustness

Domain randomization

Randomize bin placement, camera pose, lighting, object pose, and colors to reduce memorized shortcuts.

Learning

VLA fine-tuning

Move from simplified instruction/color conditioning toward a lightweight language-conditioned action model.

Open questions I am still testing

How much bin randomization is enough before the policy must use vision? Does a wrist camera improve grounding or simply increase training cost? How should action chunking or flow-matching action heads be compared against a compact MLP baseline? These questions are the motivation for keeping the behavior cloning setup simple and diagnosable first.

When Behavior Cloning Cheats.

A simple assignment became a controlled robot-learning experiment.

Sort the colored cube into the matching bin.

Vision + robot state

Instruction / color

Joint deltas

First, build an expert that can solve the task by construction.

The dataset is the experiment.

A compact multimodal policy.

The policy worked — until the bins moved.

Initial success

Generalization failure

Randomize the expert data, not just the evaluation.

Fixed layout

Randomized bins

Four clips tell the whole story.

Scripted oracle

Initial behavior cloning

Randomized-bin failure

Randomized data collection

From behavior cloning baseline toward VLA-style manipulation.

Wrist camera

Domain randomization

VLA fine-tuning

See the code, report, and demos.