Ordering by a string column but coercing it to a number for ordering

I have a column that mostly contains numbers with only a very few exceptions and want to apply order by, but given the column is of type TEXT the ordering is of course 1, 10, 100, 101, ...
Any suggestions how to coerce the column for ordering? Probably applying the sql tag with an ABS or so?

At the moment I am using this

.orderBy(sql`abs(${cardsTable.number})`)

and it seems to work so far