2011 AMC 10A Problem 3

The square brackets mean "average of two numbers" and the curly brackets mean "average of three numbers". Like ordinary parentheses, the innermost ones get worked out first.

1. The inner curly bracket. $\{1\ 1\ 0\}$ is the average of $1$, $1$ and $0$:
$$\frac{1 + 1 + 0}{3} = \frac{2}{3}.$$
2. The inner square bracket. $[0\ 1]$ is the average of $0$ and $1$:
$$\frac{0 + 1}{2} = \frac{1}{2}.$$
3. The outer curly bracket. Now the expression is $\{\tfrac{2}{3}\ \tfrac{1}{2}\ 0\}$, the average of three numbers. Add them using the common denominator $6$:
$$\frac{2}{3} + \frac{1}{2} + 0 = \frac{4}{6} + \frac{3}{6} = \frac{7}{6}.$$
4. There are three numbers, so divide by $3$:
$$\frac{7}{6} \div 3 = \frac{7}{18}.$$

The answer is $\boxed{\textbf{(D)}\ \frac{7}{18}}$.

A made-up symbol is just a rule for combining numbers. When symbols are nested, you handle them like nested parentheses: innermost first. The only thing to keep track of is how many numbers each average has, two or three, because that is what you divide by.