Direction-Scale Decomposition in Action Representation: Rethinking What to Tokenize for Vision-Language-Action Models
DSD separates direction from scale before tokenization, exposing a shared geometric convention across collection speeds, normalization statistics, and robot datasets.
Action representation matters before tokenization begins
Action representation plays a central role in discrete-token vision-language-action (VLA) learning but remains underexamined. Under conventional pose-increment representations, action tokens are sensitive to execution speed and dataset-specific normalization, potentially obscuring geometric structure shared across demonstrations and datasets. We introduce Direction-Scale Decomposition (DSD), an action representation that decomposes translation and rotation increments into direction and scale components before tokenization. DSD isolates motion direction while retaining magnitudes in separate scale channels. We evaluate DSD with uniform binning (BIN) and BEAST, a B-spline-based tokenizer, in simulation and real-world manipulation under both single-dataset and mixed-dataset training. On LIBERO, DSD improves average success rates with both tokenizers. On SimplerEnv, DSD-BIN outperforms BIN by 10.3 percentage points in overall success rate under mixed-dataset training. Real-robot experiments further show gains both with and without robotics pretraining. These results support DSD as an effective action representation for discrete-token VLA models and suggest its potential to mitigate performance degradation when training on large and diverse dataset mixtures.
Motivation
Three problems of raw action tokenization
Play or scrub each story. The controls expose how speed and normalization can change token targets—or the decoded 3-D motion— even when the intended action pattern is shared.
01
Speed changes the tokens, not the path
The same motion collected at different teleoperator speeds has the same geometry but different per-step displacement. Raw action components therefore move between token bins.
DSD comparisonThe same direction-token sequence is preserved while scale is isolated.
One path · two collection speedsIllustrative bins update live
Raw ΔEEFtokens shift
slow#135 · #130 · #134
fast#140 · #131 · #138
DSDsame sequence
slowdir #222 · #151 · #211 · scale #23
fastdir #222 · #151 · #211 · scale #37
02
Normalization bounds change the target
The same physical action can be assigned to different bins when the lower and upper normalization statistics change. The inconsistency is introduced before tokenization.
Fixed physical actionp = (4, 4, 3) mm
Same action · changing statisticsIndependent bounds
xpx = 4 mm
lower −4.0 mmupper +9.9 mm
normalized0.15
ypy = 4 mm
lower −5.0 mmupper +11.2 mm
normalized0.11
zpz = 3 mm
lower −3.0 mmupper +10.6 mm
normalized−0.12
Raw ΔEEFcomponent shifts
Raw delta end-effector normalized values and bin tokens
axis
normalized
bin token
x
0.15
#147
y
0.11
#142
z
−0.12
#112
DSDdirection shared
nx direction0.62→token #207
ny direction0.62→token #207
nz direction0.47→token #187
Scale magnitude · 6.40 mm#163Illustrative scale statistics: 0–10.0 mm. The scale token changes with these statistics while the direction tokens stay fixed.
All three direction components are tokenized. Their token IDs stay fixed while the raw-coordinate bounds move.
03
Target denormalization can distort motion
A token learned with Bridge normalization statistics can be decoded at inference with different Berkeley-UR5 statistics. Because the offsets and ranges differ by axis, this mismatch can rotate—or even reverse—the reconstructed 3-D motion while the predicted normalized action stays fixed.
Learned stats ≠ decoding stats0.00° raw direction change · DSD 0°
DSD converts each 3-D translation into a magnitude and unit direction, and each relative rotation into an angle and unit axis. The gripper action remains unchanged.
Raw action7 components[Δpₓ, Δpᵧ, Δpᶻ, Δroll, Δpitch, Δyaw, g]
TokenizerDiscrete tokens256 bins per continuous coordinate
3-D translation
Magnitude
sₚ = ‖p‖₂
Unit direction
nₚ = p / sₚ if sₚ > 0; e₃ otherwiseRelative rotation
Convert RPY to axis-angle
R = Rz(Δyaw) Ry(Δpitch) Rx(Δroll) → ω
Angle and unit axis
sᵣ = ‖ω‖₂; nᵣ = ω / sᵣ if sᵣ > 0; e₃ otherwise
Why it works
Positive scale changes preserve direction
DSD makes this invariance explicit. A longer or shorter displacement changes the scale channel, while its unit direction stays the same.
same 3-D direction
Shorter displacementsmaller scale
Direction tokensame directionunchanged
Scale tokenshorter motionsmaller
Longer displacementlarger scale
Direction tokensame directionunchanged
Scale tokenlonger motionlarger
Motion length changes only the scale token. The direction token remains identical.
Simulation
DSD improves two tokenizers and cotraining
LIBERO · 500 trials per suite
Average success
+3.8 / +3.9
BIN88.5%
DSD-BIN92.3%
BEAST88.0%
DSD-BEAST91.9%
SimplerEnv · 300 matched trials
Cotraining
+10.3 points
Bridge-only→Cotraining
BINraw action
Bridge-only: 40.0Cotraining: 33.0
−7.0
DSD-BINours
Bridge-only: 40.7Cotraining: 43.3
+2.7
Cotraining head-to-head43.3 vs 33.0+10.3 points
Cotraining lowers BIN by 7.0 points but raises DSD-BIN by 2.7 points. Under cotraining, DSD-BIN beats BIN by 10.3 points (95% CI 5.0–15.7; adjusted p = .003).
Light tracking experiment
Best rollout shown for each method
Fifty scripted demonstrations use five velocities with ten repetitions each. Each method is evaluated on ten rollouts; videos use the same crop and play at 1×.
BIN81.08DSD-BIN95.38BEAST85.41DSD-BEAST91.62
Best-score gain: DSD-BIN +14.30 vs. BINDSD-BEAST +6.21 vs. BEASTMean DTW: −20.5% / −10.1%
Real robot experiments
Inspect every rollout one by one
Choose a task and method to view one large, complete rollout at 1×. Every clip uses the same aspect-preserving crop. Success rates are measured over 30 trials per method and task.
1× · complete rollout
Cube stackingBEAST
Success rate50.0%
Without robotics pretraining60.0% → 73.3%+13.3 points
With robotics pretraining65.0% → 76.7%+11.7 points
Takeaways
A representation change with no model redesign
01
Separate direction and scale before tokenization.
02
Preserve directional structure across speeds and datasets.
03
Integrate DSD with existing tokenizers and VLA models.