C#•3y ago

✅ Serializing XML with diacritics/accents

Hi, I'm trying to serialize a class to xml that contains some strings, some of which may contain an accent (example:

genérica

genérica

) or other latin characters (example:

diseño

diseño

). Currently I'm serializing it using the following code:

private static byte[] SerializeXml<T>(T data) {
    using MemoryStream mem = new();
    using StreamWriter writer = new(mem);
    // XmlWriterSettings settings = new() {Encoding = Encoding.Default};
    
    XmlSerializer serializer = new(typeof(T));
    // using StringWriter writer = new Utf8StringWriter();
    using XmlWriter xmlWriter = XmlWriter.Create(writer);
    serializer.Serialize(xmlWriter, data);
    return mem.ToArray();
}

private static byte[] SerializeXml<T>(T data) {
    using MemoryStream mem = new();
    using StreamWriter writer = new(mem);
    // XmlWriterSettings settings = new() {Encoding = Encoding.Default};
    
    XmlSerializer serializer = new(typeof(T));
    // using StringWriter writer = new Utf8StringWriter();
    using XmlWriter xmlWriter = XmlWriter.Create(writer);
    serializer.Serialize(xmlWriter, data);
    return mem.ToArray();
}

(I was previously trying to use a StringWriter but it also doesn't work)
It ends up replacing the characters with

??

??

instead of writing the actual characters.
The final XML needs to be formatted to UTF8 (and include

encoding="utf-8"

encoding="utf-8"

at the top of the xml), so I'm not able to use iso-8859-1 (but even when I tried to use it as an encoding it replaced characters with "??")

jcotton42•7/5/23, 2:58 AM

my guess is the default encoding for XmlWriter isn't any form of unicode

jcotton42•7/5/23, 2:59 AM

https://learn.microsoft.com/en-us/dotnet/api/system.xml.xmlwritersettings.encoding?view=net-7.0

XmlWriterSettings.Encoding Property (System.Xml)

Gets or sets the type of text encoding to use.

jcotton42•7/5/23, 2:59 AM

hm, but it's UTF8 by default in .NET 7

jcotton42•7/5/23, 2:59 AM

what runtime version are you on?

Chik3rOP•7/5/23, 3:00 AM

net 7

jcotton42•7/5/23, 3:01 AM

what program are you using to view the output XML?

Chik3rOP•7/5/23, 3:01 AM

I'm currently printing it to console, but I initially checked it on a browser

Chik3rOP•7/5/23, 3:02 AM

I'm also checking the byte array and it shows 2 '?' characters

jcotton42•7/5/23, 3:02 AM

hmm

jcotton42•7/5/23, 3:02 AM

lemme try that code on my end

Chik3rOP•7/5/23, 3:03 AM

jcotton42•7/5/23, 3:05 AM

hm, it works here

jcotton42•7/5/23, 3:05 AM

also .NET 7

jcotton42•7/5/23, 3:05 AM

what if you try explicitly specifying the Encoding as UTF8 on the writer settings?

Chik3rOP•7/5/23, 3:06 AM

like this?

it still doesn't work

jcotton42•7/5/23, 3:06 AM

have you tried printing out the values in the object you're serializing
maybe they're already messed up?

Chik3rOP•7/5/23, 3:09 AM

I tried setting it to a test value () but it's still replacing it

jcotton42•7/5/23, 3:11 AM

hm
can you produce a minimal example?

Chik3rOP•7/5/23, 3:37 AM

I discovered the issue. I have a class to surround anything with (where

is the class name of the generic

) and it was done like this

It serialized the data so that the element name changes with the class name, but I made it use ascii instead of utf8, causing characters to be replaced with

. It's fixed now by changing to , although I should probably check if there's a better/correct way to do this

Chik3rOP•7/5/23, 3:37 AM

Thanks for the help