Both the BMP format and the TIFF format are limited to a maximum file size of 4 GBytes. When these file formats were developed no one could imagine a need to support larger files but modern imaging machines must support panels of 3 x 3 meters with pixel sizes of less than 1 um or 300 x 300 mm regions with pixel size of 0.01 um.
Header LimitationThe basic bitmap file structure is not the cause of this 4GB limit. Instead, there is a header in both file formats that informs any parser the size of the file. The bytes used for this header can only represent a maximum of 4 GBytes.
One of Artwork's clients wanted to use gbr_rip but also needed to produce BMP style bitmaps that were as large as 50 GByte. What to do? These files were so large because monochrome BMP does not support compression.
Solution - Ignore the HeaderWe agreed that the BMP file size information in the header was not really necessary (since a program could compute this after reading the file) and that we would simply set those four bytes = 0. We went ahead and provided the rest of the data in BMP format and were able to support these 50 GBytes files.
The obvious implication of producing such as BMP file is that most BMP parsers won't read it. They will likely balk when they come to the header section and see that the file size = 0. Of course, our client modified his parser to support such files.
The same problem and a similar solution can be applied to the TIFF format. Fortunately, because TIFF supports packbits compression, we run into the limit less often. Nevertheless a number of companies have agreed on a way to go past the 4 GB limit. The -bigtiff commmand line option enables this behavior. We decided to follow the same header as defined by Aware Systems. For more details see: https://www.awaresystems.be/imaging/tiff/bigtiff.html
In order to enable our clients to view and analyze such large files, we have upgraded Artwork's Very Large Bitmap Viewer (VLBV) to support these files. We've also added optimizations to the loading/compressing approach.