extend reporting of broken Content-Type headers

[albrechtd]

I recently got several (malspam) messages with broken Content-Type headers like

Content-Type: text/Here comes some crap

GMime reports the content type as text/Here, but does not report the header as being invalid. Using the attached patch detect-broken-content-type.patch, the warning is emitted as expected (I guess there is a plethora of other, yet undetected, cases, though…).

[jstedfast]

Amusingly, I've been working on similar stuff professionally (I work on Microsoft Exchange server these days).

I'm also trying to make the time to add this type of detection to MimeKit, which, these days, is where I am putting all of my free time. I don't really like the API in Exchange (it existed before I joined) and this has reminded me of a different approach that I like better.

Too bad you aren't using C# - I would argue that MimeKit is a better parser than GMime at this point (and likely faster as well thanks to a lot of the hardware acceleration I've been pushing into MimeKit).

Is your app firmly dependent on GMime being in C?

[jstedfast]

Just rebased my dev branch for this type of feature in MimeKit: jstedfast/MimeKit@945a46e

This is the secret sauce for detecting 8bit headers and embedded null bytes in headers/content: https://github.com/jstedfast/MimeKit/blob/945a46e2da75718e07dd59e74dd9d8a47dbfc027/MimeKit/Utils/Memory.cs

This allows MimeKit's parser to detect this stuff while its looking for the end of a line whereas GMime's implementation currently loops looking for the end of the line and then rescans to see if the header value text is 8bit.

GMime also doesn't currently detect null bytes which is something that it probably should detect :-\

[albrechtd]

[snip]

Too bad you aren't using C# - I would argue that MimeKit is a better parser than GMime at this point (and likely faster as well thanks to a lot of the hardware acceleration I've been pushing into MimeKit).

Is your app firmly dependent on GMime being in C?

Apart from Balsa which doesn't make any use of detecting anomalies (as reporting them is probably confusing for the user; too many legitimate applications produce half-broken mails), I use this library and in particular the anomaly detection in several somehow “forensic” tools on Linux and BSD. Re-writing them is clearly no option.

However, an interface for a standard language (C, maybe C++ or Rust) might be interesting for looking into proprietary components like TNEF, as koodaamo's parser is incomplete and has several flaws. As we want to get rid of any critical component which we can't audit and control completely (source code, “telemetry” data transmission, etc.) or operate in fully isolated (air-gap) environment, the importance is not extremely high, though.

GMime also doesn't currently detect null bytes which is something that it probably should detect :-\

No real issue IMHO – NUL is not allowed by RFC 5321 (if I read the standard correctly) and detected by Suricata & friends.