Task reference

Main task families

ID#	Level	Task Variant	Dist.	Success Criterion
Task 1: Put Bread on Plate
1	Easy	Put A bread on the red plate	ID	Bread is resting on the correct colored plate, and SO-101 moved back to its home position.
2	Easy	Put B bread on the blue plate	ID
3	Medium	Put A bread on the blue plate	ID
4	Medium	Put B bread on the red plate (1 distractor)	ID
5	Hard	Put A bread on the red plate (2 distractors)	ID
6	Easy	Put C bread on the red plate	OOD
7	Easy	Put C bread on the blue plate	OOD
8	Medium	Put A bread on the yellow plate	OOD
9	Medium	Put B bread on the yellow plate	OOD
10	Hard	Put C bread on the yellow plate	OOD
Task 2: Put Bowl on Coaster
11	Easy	Put red bowl on green coaster	ID	Correct bowl is touching/resting on correct colored coaster, and SO-101 moved back to its home position.
12	Easy	Put blue bowl on orange coaster	ID
13	Medium	Put blue bowl on green coaster	ID
14	Medium	Put red bowl on purple coaster (1 distractor)	ID
15	Hard	Put yellow bowl on purple coaster (2 distractors)	ID
16	Easy	Put blue bowl on purple coaster	OOD
17	Easy	Put red bowl on orange coaster	OOD
18	Medium	Put yellow bowl on green coaster	OOD
19	Medium	Put yellow bowl on orange coaster	OOD
20	Hard	Put green bowl on yellow coaster	OOD
Task 3: Stack Blocks
21	Easy	Stack red block on blue block	ID	Correctly colored top block has touched/rested on correctly colored bottom block for more than 2 seconds.
22	Easy	Stack yellow block on blue block	ID
23	Medium	Stack blue block on yellow block	ID
24	Medium	Stack blue block on red block (1 distractor)	ID
25	Hard	Stack red block on yellow block (2 distractors)	ID
26	Easy	Stack yellow block on red block	OOD
27	Easy	Stack blue block on blue block	OOD
28	Medium	Stack red block on green block	OOD
29	Medium	Stack green block on yellow block	OOD
30	Hard	Stack green block on green block	OOD
Task 4: Fold Towel
31	Easy	Fold pink towel in half	ID	Correctly colored towel's edges are lifted and folded on itself by more than 50%, and SO-101 moved back to its home position.
32	Easy	Fold yellow towel in half	ID
33	Medium	Fold pink towel in half (1 distractor)	ID
34	Medium	Fold yellow towel in half (1 distractor)	ID
35	Hard	Fold pink towel in half (2 distractors)	ID
36	Easy	Fold blue towel in half	OOD
37	Easy	Fold blue towel in half	OOD
38	Medium	Fold blue towel in half (1 distractor)	OOD
39	Medium	Fold blue towel in half (2 distractors)	OOD
40	Hard	Fold blue towel in half (2 distractors)	OOD
Task 5: Open Oven (No OOD Variants)
41	Easy	Open the oven	ID	Oven door opened for 2+ seconds, and SO-101 moved back to its home position.
42	Easy	Open the oven	ID
43	Medium	Open the oven (1 distractor)	ID
44	Medium	Open the oven (2 distractors)	ID
45	Hard	Open the oven (2 distractors)	ID
Task 6: Clean Whiteboard (No OOD Variants)
46	Easy	Clean whiteboard with eraser	ID	Eraser wiped whiteboard 2 or more times, and then placed next to whiteboard.
47	Easy	Clean whiteboard with eraser	ID
48	Medium	Clean whiteboard with eraser (1 distractor)	ID
49	Medium	Clean whiteboard with eraser (2 distractors)	ID
50	Hard	Clean whiteboard with eraser (2 distractors)	ID
Task 7: Pour Pepper
51	Easy	Pour 1 shake of pepper into the red plate	ID	The exact number of pepper shakes is poured into the correct colored plate, the pepper is placed to the left of the plate, and SO-101 moved back to its home position.
52	Easy	Pour 3 shakes of pepper into the red plate	ID
53	Medium	Pour 2 shakes of pepper (1 distractor)	ID
54	Medium	Pour 1 shake of pepper (1 distractor)	ID
55	Hard	Pour 3 shakes of pepper (2 distractors)	ID
56	Easy	Pour 4 shakes of pepper into red plate	OOD
57	Easy	Pour 5 shakes of pepper into red plate	OOD
58	Medium	Pour 1 shake of pepper into blue plate	OOD
59	Medium	Pour 3 shakes of pepper into blue plate	OOD
60	Hard	Pour 5 shakes of pepper into blue plate	OOD
Task 8: Lift Bowl
61	Easy	Lift green bowl one time	ID	Correct colored bowl lifted up and down the correct number of times, and SO-101 moved back to its home position.
62	Easy	Lift blue bowl one time	ID
63	Medium	Lift red bowl three times	ID
64	Medium	Lift green bowl three times (1 distractor)	ID
65	Hard	Lift blue bowl three times (2 distractors)	ID
66	Easy	Lift green bowl two times	OOD
67	Easy	Lift blue bowl two times	OOD
68	Medium	Lift red bowl two times	OOD
69	Medium	Lift yellow bowl two times	OOD
70	Hard	Lift yellow bowl four times	OOD
Task 9: Press Button
71	Easy	Press button one time	ID	Button is touched the correct number of times, and SO-101 moved back to its home position.
72	Easy	Press button three times	ID
73	Medium	Press button one time (1 distractor)	ID
74	Medium	Press button three times (1 distractor)	ID
75	Hard	Press button three times (2 distractors)	ID
76	Easy	Press button two times	OOD
77	Easy	Press button four times (1 distractor)	OOD
78	Medium	Press button two times (2 distractors)	OOD
79	Medium	Press button four times (2 distractors)	OOD
80	Hard	Press button five times (2 distractors)	OOD
Task 10: Collect Blocks
81	Easy	Collect 2 blocks into blue box	ID	All the blocks on the workspace are placed into the correct colored box, and SO-101 moved back to its home position.
82	Easy	Collect 2 blocks into yellow box	ID
83	Medium	Collect 3 blocks into blue box (1 distractor)	ID
84	Medium	Collect 3 blocks into yellow box	ID
85	Hard	Collect 4 blocks into blue box (2 distractors)	ID
86	Easy	Collect 2 blocks into pink box	OOD
87	Easy	Collect 3 blocks into pink box	OOD
88	Medium	Collect 5 blocks into blue box	OOD
89	Medium	Collect 5 blocks into yellow box	OOD
90	Hard	Collect 5 blocks into pink box	OOD

ID and OOD variants

Appendix C contains the full 90-variant list. It includes five ID variants and five OOD variants for most tasks, with no OOD variants for Tasks 5 and 6.

Reference scene images

Appendix E contains the 90 reference images used to set up the evaluation scenes. Use these images to match the task layout before starting the policy.