A couple posts ago, I talked about the word data: how it feels like a substance rather than a plural,1 and my assertion that it's okay to say the data is just as you might say the rice is. But maybe I should have asked instead: why don't we say datums?
I don't know how many aquaria or stadia you've been to, but I would bet that you have been to a couple of aquariums or stadiums. And it's possible that you have designed or used database schemata, but you might have thought of them as schemas (as you should).
We often pluralize loan words according to the rules of English, but not always. I've never heard anyone say they were going out to get a couple pizze, for example, yet we use data, criteria, phenomena, and sometimes even indices. Even worse, you'll sometimes see people using phenomena, media, and criteria as singular nouns out in the wild.
I'm a big fan of etymology, but you just can't force English to be Latin or Greek. And even if you could, you would have to fight centuries of people getting it wrong. For everyone who waggles a finger and corrects syllabuses to syllabi there's an annoying fact to contradict them.
We must get used to inconsistency: no one says viri when talking about viruses or piani when talking about pianos, and no one says bacterias, algas,2 or phenomenons either, for no good reason. And then there are words like ignoramus: we think the Latin plural ignorami is obvious but we are dead wrong—because we do not know3 that in Latin ignoramus is not a noun.
No one said the English language was easy.
A mass noun rather than a count noun, in other words.
Alga is the singular of algae.
See what I did there?
Not easy. I heard that Harvard has stopped using Latin on its diplomas (diplomae?) An architectural firm I worked for a few years ago wanted to include a brief resume of mine in a proposal they were sending out, and asked me to read and approve it. They described me as an "alumni of Yale University". I could only say that there was only one of me, and maybe they should just say "graduate" instead. I went to Yale, after all....
re: 3 - I saw what you did there, but I did have to Google it.