The Bubby (יוכבד) was counted when she was a baby, so Shevet Levi are counted when they are babies. Does that make sense?