Q24: Why should string_decoder be used instead of buffer.toString() when parsing multi-byte streams?
When streaming multi-byte UTF-8 encoded characters (e.g., emojis or non-Latin scripts), a data chunk boundary may split a multi-byte character sequence in half. Converting chunks using buffer.toString() outputs corrupt character symbols (``). The string_decoder module preserves incomplete multi-byte sequences at chunk boundaries until the next chunk arrives.
Key Takeaways
buffer.toString()corrupts multi-byte UTF-8 characters split across stream chunk boundaries.StringDecoderbuffers incomplete multi-byte sequences until remaining bytes arrive.
Knowledge Check
What issue occurs when using buffer.toString('utf8') on raw stream chunks containing split multi-byte characters?