Short Answer:
If you want to appear "properly informed" for what passes for the moment as educated, use exes (no matter how funny it looks1). Otherwise, feel free to use ex's. Just don't get it confused with the possessive of the singular ex, which is also ex's.
If only English were/was (take your pick) so easy...then the plural of the two letter word ox would be oxes, for everyone.
Yours (or, for some, your's) is a great question, since the word ex meaning ex-girlfriend is a detached prefix. It looks weird standing there so alone, so short, ending with -x, added to words with the hyphen attached: ex-girlfriend, ex-convict.
Ex is also the "name" of the letter X, and people don't agree how to pluralize letters. Answers to the ELU question What is the proper way to write the plural of a single letter? return mixed results. Yet, if one decides that the plural of x is x's, it's not unreasonable to intuit that the plural of ex is ex's.
This plural-marking apostrophe carries over to other short words. What's the plural of and? Is it ands or and's? You will find plenty of the latter in the idiom no if's and's or but's, spelled like that, the apostrophe indicating plural.
Sure, there are rules devised by grammarians, but these are the same people who brought you such damnable rules as don't start a sentence with and or end one with a preposition.
Most native speakers do not intuitively know how to pluralize ex. There are plenty of examples of pluralizing it as ex's: for example, see the reference to the hit single "All My Ex's Live in Texas" below.
This doubt carries over to certain words that end in -ex. You'll find plenty of T-Rex's running around.
And this is not limited to ex. People, native speakers, tend to reach for the apostrophe when in doubt. Sometimes last names are pluralized with an apostrophe: the Jackson's. (And no one can authoritatively say it's wrong to do so.) Two hundred years ago, the apostrophe was used to pluralize "foreign-sounding" words ending with a vowel: pasta's, potato's, ouzo's (for a reference on this, see the next link). Now if a grocer uses potato's or banana's he's accused of using the dreadful greengrocers’ apostrophe.
In any case, the writer of the hit single "All My Ex's Live in Texas" made a stylistic choice (consciously or not), and I'm not sure that this spelling (ex's) was necessarily done out of ignorance of "standard" pluralization rules; or whether it was chosen as an assertion of an alternative spelling or just because it looked better.
Apparently the author of this tweet was using their/his (take your pick) noggin:
#ifwedate dont bring up ya old hoes or ex's i'll smoove cut you off as fast as pnd posted that pic with kehlani in his bed
Notice how careful 'brown boy' was to pluralize ho as hoes, but good luck determining if that's "correct." Hos has a history.
My answer is that it depends. Why shouldn't it be? This ELU site & ELL usually eschew prescriptivist answers. And if you can take your pick in the case of other uses, why can't you take your pick in the case of your ex's/exes? As usual, you should choose the Style Guide of your choice whether that's Strunk & White or Shafer & Strait. If English speakers acted in accord, it would be oxes; we've had over a millennium to conform this beast to the norm.
Last thought: perhaps it should be ices, given that ex is from the Latin and the traditional plural of codex is codices, index is indices, etc. Many might feel ices to be quite an appropriate plural for ex.
1 "The plural of ex is exes, and the possessive is ex's—but be aware that many readers will find these forms odd-looking." (The Oxford Dictionary of American Usage and Style)
