Require support for BMP files

[villagemedia]

We are attempting to move from ImageResizer to ImageFlow. We noticed a lot of errors after switching, and it seems that ImageFlow does not support bitmaps. We have user submitted content that has a .jpg extension, but it's actually a bitmap. These resize just fine in ImageResizer, but fail in ImageFlow.

ImageflowException: ImageMalformed: NoEnabledDecoderFound: No ENABLED decoder found for file starting in [42, 4D, 36, C4, 6D, 1, 0, 0, 0, 0, 36, 0] at
imageflow_core\src\codecs\mod.rs:155:20

imageflow/imageflow_core/src/codecs/mod.rs

Line 155 in 22a2bf9

return Err(nerror!(ErrorKind::NoEnabledDecoderFound, "No ENABLED decoder found for file starting in {:X?}", bytes))

imageflow_core\src\context.rs:194:103

imageflow/imageflow_core/src/context.rs

Line 194 in 22a2bf9

let codec_value = CodecInstanceContainer::create(self, io, io_id, direction).map_err(|e| e.at(here!()))?;

imageflow_core\src\context.rs:235:66

imageflow/imageflow_core/src/context.rs

Line 235 in 22a2bf9

self.add_io(io, io_id, IoDirection::In).map_err(|e| e.at(here!()))

imageflow_abi\src\lib.rs:703:62

imageflow/imageflow_abi/src/lib.rs

Line 703 in 22a2bf9

c.add_input_buffer(io_id,bytes).map_err(|e| e.at(here!()))?;

[lilith]

It's exceptionally rare for DIBs to be used (you're the first), but I suppose it's not a complicated codec to implement. Do you have a support plan with us?

…

On Tue, Feb 21, 2023, 2:11 PM villagemedia ***@***.***> wrote: We are attempting to move from ImageResizer to ImageFlow. We noticed a lot of errors after switching, and it seems that ImageFlow does not support bitmaps. We have user submitted content that has a .jpg extension, but it's actually a bitmap. These resize just fine in ImageResizer, but fail in ImageFlow. ImageflowException: ImageMalformed: NoEnabledDecoderFound: No ENABLED decoder found for file starting in [42, 4D, 36, C4, 6D, 1, 0, 0, 0, 0, 36, 0] at imageflow_core\src\codecs\mod.rs:155:20 https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_core/src/codecs/mod.rs#L155 imageflow_core\src\context.rs:194:103 https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_core/src/context.rs#L194 imageflow_core\src\context.rs:235:66 https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_core/src/context.rs#L235 imageflow_abi\src\lib.rs:703:62 https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_abi/src/lib.rs#L703 — Reply to this email directly, view it on GitHub <#626>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2LH5KG5VBYZOM3VDH26TWYUVR3ANCNFSM6AAAAAAVDQZKYI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[villagemedia]

No, we don't, but we've been a client for a long time! On Tue, Feb 21, 2023 at 5:09 PM Lilith River ***@***.***> wrote:

…

It's exceptionally rare for DIBs to be used (you're the first), but I suppose it's not a complicated codec to implement. Do you have a support plan with us? On Tue, Feb 21, 2023, 2:11 PM villagemedia ***@***.***> wrote: > We are attempting to move from ImageResizer to ImageFlow. We noticed a lot > of errors after switching, and it seems that ImageFlow does not support > bitmaps. We have user submitted content that has a .jpg extension, but it's > actually a bitmap. These resize just fine in ImageResizer, but fail in > ImageFlow. > > ImageflowException: ImageMalformed: NoEnabledDecoderFound: No ENABLED > decoder found for file starting in [42, 4D, 36, C4, 6D, 1, 0, 0, 0, 0, 36, > 0] at > imageflow_core\src\codecs\mod.rs:155:20 > > https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_core/src/codecs/mod.rs#L155 > imageflow_core\src\context.rs:194:103 > > https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_core/src/context.rs#L194 > imageflow_core\src\context.rs:235:66 > > https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_core/src/context.rs#L235 > imageflow_abi\src\lib.rs:703:62 > > https://github.com/imazen/imageflow/blob/22a2bf99f278d4dd7b8da65ee1604b5f84eb1d50/imageflow_abi/src/lib.rs#L703 > > — > Reply to this email directly, view it on GitHub > <#626>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAA2LH5KG5VBYZOM3VDH26TWYUVR3ANCNFSM6AAAAAAVDQZKYI > > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > — Reply to this email directly, view it on GitHub <#626>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT5Y6H3ZXQE6QI5RYVZAM6DWYU4LNANCNFSM6AAAAAAVDQZKYI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[lilith]

I've moved this to the Imageflow project for consideration. I typically don't implement features unless there are multiple users needing them, but I'll keep it in mind. For now, I would suggest normalizing your bitmaps to .png.

[villagemedia]

I would love to normalize to png or jpg, but the problem is that we have public (uncontrolled) users submitting images. Until this experience, I wasn't aware that some users were submitting bmp masquerading as jpg. Regardless, thank you for your consideration!

…

On Fri, Feb 24, 2023 at 12:21 AM Lilith River ***@***.***> wrote: I've moved this to the Imageflow project for consideration. I typically don't implement features unless there are multiple users needing them, but I'll keep it in mind. For now, I would suggest normalizing your bitmaps to .png. — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT5Y6H5GIS47QBBFKJBNVATWZBANDANCNFSM6AAAAAAVGPJYP4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[lilith]

I remember at one point Paint defaulted to .bmp, but is that still occurring with new incoming files? Some upload tools can check the first few bytes - the file signature - and can reject invalid formats. We have a C# method for this on the server side as well.

…

On Fri, Feb 24, 2023, 6:09 AM villagemedia ***@***.***> wrote: I would love to normalize to png or jpg, but the problem is that we have public (uncontrolled) users submitting images. Until this experience, I wasn't aware that some users were submitting bmp masquerading as jpg. Regardless, thank you for your consideration! On Fri, Feb 24, 2023 at 12:21 AM Lilith River ***@***.***> wrote: > I've moved this to the Imageflow project for consideration. I typically > don't implement features unless there are multiple users needing them, but > I'll keep it in mind. For now, I would suggest normalizing your bitmaps to > .png. > > — > Reply to this email directly, view it on GitHub > <#626 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AT5Y6H5GIS47QBBFKJBNVATWZBANDANCNFSM6AAAAAAVGPJYP4 > > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2LH4C554UHJKENDJ3XGLWZCXJRANCNFSM6AAAAAAVGPJYP4> . You are receiving this because you commented.Message ID: ***@***.***>

[villagemedia]

Shockingly yes, this happened very recently. I searched for cameras or phones that might do this, but found nothing. We will scan the first bytes and block these moving forward. No idea how many exist on our platform like this, however. On Fri, Feb 24, 2023 at 10:29 AM Lilith River ***@***.***> wrote:

…

I remember at one point Paint defaulted to .bmp, but is that still occurring with new incoming files? Some upload tools can check the first few bytes - the file signature - and can reject invalid formats. We have a C# method for this on the server side as well. On Fri, Feb 24, 2023, 6:09 AM villagemedia ***@***.***> wrote: > I would love to normalize to png or jpg, but the problem is that we have > public (uncontrolled) users submitting images. Until this experience, I > wasn't aware that some users were submitting bmp masquerading as jpg. > > Regardless, thank you for your consideration! > > On Fri, Feb 24, 2023 at 12:21 AM Lilith River ***@***.***> > wrote: > > > I've moved this to the Imageflow project for consideration. I typically > > don't implement features unless there are multiple users needing them, > but > > I'll keep it in mind. For now, I would suggest normalizing your bitmaps > to > > .png. > > > > — > > Reply to this email directly, view it on GitHub > > < #626 (comment) > >, > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AT5Y6H5GIS47QBBFKJBNVATWZBANDANCNFSM6AAAAAAVGPJYP4 > > > > . > > You are receiving this because you authored the thread.Message ID: > > ***@***.***> > > > > — > Reply to this email directly, view it on GitHub > <#626 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAA2LH4C554UHJKENDJ3XGLWZCXJRANCNFSM6AAAAAAVGPJYP4 > > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT5Y6H6C4TIWJC2D6FMVWC3WZDHXBANCNFSM6AAAAAAVGPJYP4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[lilith]

Imazen.Common.FileTypeDetection should be useful here: https://github.com/imazen/imageflow-dotnet-server/blob/main/src/Imazen.Common/FileTypeDetection/FileTypeDetector.cs

…

On Fri, Feb 24, 2023, 8:31 AM villagemedia ***@***.***> wrote: Shockingly yes, this happened very recently. I searched for cameras or phones that might do this, but found nothing. We will scan the first bytes and block these moving forward. No idea how many exist on our platform like this, however. On Fri, Feb 24, 2023 at 10:29 AM Lilith River ***@***.***> wrote: > I remember at one point Paint defaulted to .bmp, but is that still > occurring with new incoming files? > > Some upload tools can check the first few bytes - the file signature - and > can reject invalid formats. We have a C# method for this on the server side > as well. > > On Fri, Feb 24, 2023, 6:09 AM villagemedia ***@***.***> wrote: > > > I would love to normalize to png or jpg, but the problem is that we have > > public (uncontrolled) users submitting images. Until this experience, I > > wasn't aware that some users were submitting bmp masquerading as jpg. > > > > Regardless, thank you for your consideration! > > > > On Fri, Feb 24, 2023 at 12:21 AM Lilith River ***@***.***> > > wrote: > > > > > I've moved this to the Imageflow project for consideration. I typically > > > don't implement features unless there are multiple users needing them, > > but > > > I'll keep it in mind. For now, I would suggest normalizing your bitmaps > > to > > > .png. > > > > > > — > > > Reply to this email directly, view it on GitHub > > > < > #626 (comment) > > >, > > > or unsubscribe > > > < > > > https://github.com/notifications/unsubscribe-auth/AT5Y6H5GIS47QBBFKJBNVATWZBANDANCNFSM6AAAAAAVGPJYP4 > > > > > > . > > > You are receiving this because you authored the thread.Message ID: > > > ***@***.***> > > > > > > > — > > Reply to this email directly, view it on GitHub > > < #626 (comment) > >, > > or unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAA2LH4C554UHJKENDJ3XGLWZCXJRANCNFSM6AAAAAAVGPJYP4 > > > > . > > You are receiving this because you commented.Message ID: > > ***@***.***> > > > > — > Reply to this email directly, view it on GitHub > <#626 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AT5Y6H6C4TIWJC2D6FMVWC3WZDHXBANCNFSM6AAAAAAVGPJYP4 > > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2LH3OUPCCZQDCTBR4GXLWZDH6XANCNFSM6AAAAAAVGPJYP4> . You are receiving this because you commented.Message ID: ***@***.***>

[bes-internal]

Don't take it as rudeness, but I thought imageflow was positioning itself as a replacement for imagemagick, but it turned out to be an enterprise server with a support plan...
Our users also send anything to the server: what is displayed on their systems with a preview. This is a wild world and only on servers is it normalized.
So, for good reason, the question on the agenda is not "why", but "when".

[lilith]

No, Imageflow doesn't aim to try to parse every image format or implement every feature. The focus is to provide very secure and very performant image processing for key web image formats, with exceptionally good file compression and image quality. If you look through these 630 vulnerabilities in ImageMagick, you'll notice a pattern - they tend to be about obscure image formats: https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html

…

On Sat, Feb 25, 2023, 3:44 PM Vladimir Varlamov ***@***.***> wrote: Don't take it as rudeness, but I thought imageflow was positioning itself as a replacement for imagemagick, but it turned out to be an enterprise server with a support plan... Our users also send anything to the server: what is displayed on their systems with a preview. This is a wild world and only on servers is it normalized. So, for good reason, the question on the agenda is not "why", but "when". — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA2LHYU344IGHGYWRTTKJTWZKDMZANCNFSM6AAAAAAVGPJYP4> . You are receiving this because you commented.Message ID: ***@***.***>

[villagemedia]

We tried to move to ImageFlow again today, but had to revert back to ImageResizing again. We are still seeing loads of user-supplied images that are really bitmaps with a jpg/jpeg extension. We saw over 50,000 exceptions within about a half hour. We have considered rejecting these types of user-uploaded images, but it's not reasonable to impose this on our users as most will have no idea how to remedy the issue. So, pending a solution from ImageFlow, we will have to abandon the migration. Thanks, Scott On Sun, Feb 26, 2023 at 1:11 AM Lilith River ***@***.***> wrote:

…

No, Imageflow doesn't aim to try to parse every image format or implement every feature. The focus is to provide very secure and very performant image processing for key web image formats, with exceptionally good file compression and image quality. If you look through these 630 vulnerabilities in ImageMagick, you'll notice a pattern - they tend to be about obscure image formats: https://www.cvedetails.com/vulnerability-list/vendor_id-1749/Imagemagick.html On Sat, Feb 25, 2023, 3:44 PM Vladimir Varlamov ***@***.***> wrote: > Don't take it as rudeness, but I thought imageflow was positioning itself > as a replacement for imagemagick, but it turned out to be an enterprise > server with a support plan... > Our users also send anything to the server: what is displayed on their > systems with a preview. This is a wild world and only on servers is it > normalized. > So, for good reason, the question on the agenda is not "why", but "when". > > — > Reply to this email directly, view it on GitHub > <#626 (comment) >, > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAA2LHYU344IGHGYWRTTKJTWZKDMZANCNFSM6AAAAAAVGPJYP4 > > . > You are receiving this because you commented.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AT5Y6HY57NGNCKIU4IKWMALWZLXZZANCNFSM6AAAAAAVGPJYP4> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[lilith]

Hi Scott,

That's really unfortunate. I'm not seeing other user instances where DIBs masquerading as .jpeg files are becoming an issue, but it's clear some tool or system is producing them.

I could write a codec for these if you establish a support contract with us. On my end, it's a massive commitment since I won't be able to drop that functionality in the future, and it is part of the attack surface for the product, so it's a liability for basically everyone who doesn't need DIB support. I'd implement it in Rust, though, which should mitigate many classes of security issues.

Kind regards,
Lilith River