N
Neon11mo ago
like-gold

Answered: Unexpected behavior with pg_trgm and special characters

Hello guys. First, thanks for the awesome work. I have encountered some inconsistency regarding the pg_tgrm extension on Neon. When I use the show_trgm, it has weird behaviors when using Japanese characters: Example with no special characters: Works normally.
SELECT show_trgm('res'), show_trgm('very interesting place'), similarity('res', 'very interesting place');
# show_trgm show_trgm similarity
1 {" r"," re","es ",res} {" i"," p"," v"," in"," pl"," ve",ace,"ce ",ere,ery,est,ing,int,lac,"ng ",nte,pla,res,"ry ",sti,ter,tin,ver} 0.03846154
# show_trgm show_trgm similarity
1 {" r"," re","es ",res} {" i"," p"," v"," in"," pl"," ve",ace,"ce ",ere,ery,est,ing,int,lac,"ng ",nte,pla,res,"ry ",sti,ter,tin,ver} 0.03846154
Example with japanese characters: Skips everything in Neon.
SELECT show_trgm('面白い'), show_trgm('とても面白い場所'), similarity('面白い', 'とても面白い場所');
Neon:
# show_trgm show_trgm similarity
1 {} {} 0
# show_trgm show_trgm similarity
1 {} {} 0
My Docker PgSQL:
show_trgm show_trgm similarity
{0xb55e8b,0xc4530f,0x0b774a,0x74c091} {0x9176bd,0x920e92,0x960770,0xbe019e,0xed92a2,0xf9c014,0x14dccf,0x6fc655,0x74c091} 0.083333336
show_trgm show_trgm similarity
{0xb55e8b,0xc4530f,0x0b774a,0x74c091} {0x9176bd,0x920e92,0x960770,0xbe019e,0xed92a2,0xf9c014,0x14dccf,0x6fc655,0x74c091} 0.083333336
When I switched my staging environment from Neon to a normal PostgreSQL in Docker, it worked without any problems. Please, is there any configuration that I can do in Neon to support special and Japanese characters. Thank you in advance. Will
2 Replies
harsh-harlequin
harsh-harlequin11mo ago
Hmm, not sure tbh. This might be related to https://neon.tech/docs/reference/compatibility#collation-support
Neon
Postgres compatibility - Neon Docs
Neon is Postgres. However, as a managed Postgres service, there are some differences you should be aware of. Postgres versions Neon supports Postgres 14, 15, 16, 17. You can select the Postgres versio...
like-gold
like-goldOP11mo ago
My solution was to create the database with the encoding like listed here: https://neon.tech/docs/reference/compatibility#database-encoding Hope this helped someone. I used the encoding en_US.UTF8, like it was with my Docker environment and everything started working. Thanks @Mahmoud for the help. Hope this helps other people
Neon
Postgres compatibility - Neon Docs
Neon is Postgres. However, as a managed Postgres service, there are some differences you should be aware of. Postgres versions Neon supports Postgres 14, 15, 16, 17. You can select the Postgres versio...

Did you find this page helpful?