Yeah, black in colour-theory is technically a lack of colour, and white is the most colourful, as it’s a combination of all colours (or at least red, green and blue).

Skin colours are weird. I’m not white - I’m like a peach colour. But it is what it is, and I’ve not found better names, without sounding like a pedantic asshole.