144 lines
10 KiB
Markdown
144 lines
10 KiB
Markdown
|
### ShrinkRay/SR/.sr*
|
||
|
|
---------------------
|
||
|
|
### The What
|
||
|
|
---
|
||
|
|
This is the first entry into the ODynaMIC container format family. ODynaMIC (Open Dynamic Media Interchange Container) is intended to be an easily extendable set of media container formats that offers multi level open optimizations and easy tooling to free media from format restrictions while enabling users to continue using the systems they are familiar with.
|
||
|
|
|
||
|
|
SR is intended for artists working with pixel art and other artfully restricted mediums. This is done primarily to reduce network traffic and make transferring media over very low bandwidth networks (in our case LoRa) more feasible.
|
||
|
|
|
||
|
|
Currently it provides the .sr and .srb (ShrinkRay Battery) formats, both are for pixel art but Battery provides a frames/layer system which in turn provides an animation system
|
||
|
|
|
||
|
|
### The Why
|
||
|
|
---
|
||
|
|
Having worked with images in a serious capacity for about 15 years I've always wanted something smaller and easier to work with then existing formats. Many formats are easy to work with on the basis of an end user/power user but if you really want to alter things/make specific optimizations you'll have to write image processing code in many cases and in some cases you're simply out of luck. While languages like Processing and tools like Pillow make this relatively simple I wanted something even simpler, smaller and easier to extend/modify/etc.
|
||
|
|
### The How
|
||
|
|
---
|
||
|
|
In order to achieve the aforementioned goals we leverage some novel optimizations: index collapsing, bitflip arrays and similar frame reduction.
|
||
|
|
|
||
|
|
#### Index collapsing
|
||
|
|
---
|
||
|
|
Take a 3x3 image as an example, everything but the center middle pixel is white.
|
||
|
|
|
||
|
|
With index collapsing we auto index as we go so the first pixel would give us an index of `[(0,0,0)]` which can be reduced to `[0]` since all three numbers are the same, the pixels array at this stage would look like this `[0]` the 0 referring to the index of the color.
|
||
|
|
|
||
|
|
Stepping forward since the next pixel is also white we skip indexing, see that the last pixel was white and instead of adding another 0 we start building our first range. A range is an array of 2 values: the number of times to repeat the index reference and the index reference itself. Our pixel array now looks like this `[[2, 0]]`.
|
||
|
|
|
||
|
|
We keep stepping through the next pixels, the next two are the same so our pixels array looks like this `[[4, 0]]`.
|
||
|
|
|
||
|
|
We are now on the center middle pixel. Since this is a new color we index it. Our index now looks like `[0, 255]` (our color can be collapsed to 255 because it is (255, 255, 255), this can be reduced further to 1 with a special flag if desired). Our pixel array now looks like `[[4, 0], 1]`
|
||
|
|
|
||
|
|
All we have left is more white pixels so after we step through those this is our final index `[0, 255]` and this is our final pixel array `[[4, 0], 1, [4, 0]]`. This representation is more efficient but this is further optimized with bitflip arrays.
|
||
|
|
|
||
|
|
#### Bitflip arrays
|
||
|
|
---
|
||
|
|
While we use msgpack for other optimizations this is a place where we can optimize further then their representation can because we have less generic needs. Bitflip arrays are a bytes representation of an array which handles part of the array being 2 values. It works like this:
|
||
|
|
|
||
|
|
We step through the array and convert each int value into bytes directly, bitshift left (leaving a 0 bit at the end) and append those bytes to the result bytes. When we ecounter an array we convert the int like before and bitshift left but flip the last bit to be 1, then append it to the bytes array and add the next value of the array as we do a normal int.
|
||
|
|
|
||
|
|
When we are reading the bytes for the array we reach byte by byte and check the last bit of each byte. If it's a 0 we bitshift right, convert the bytes to an int and append that to the pixel array. If it's a 1 we bitshift right, convert the bytes to an int, then get the next byte, convert that to an int, add both to an array and append that array to the pixel array.
|
||
|
|
|
||
|
|
After this we rebuild the pixel array by using the index and values and ranges from the temporary pixel array.
|
||
|
|
|
||
|
|
Very briefly here, I want to point out that this process is in fact lossless, in the literal sense that no data is lost in any way (if you don't believe me check the code), but as you'll see later, the optimizations here can make ShrinkRay outperform existing lossless formats.
|
||
|
|
|
||
|
|
### Similar frame reduction
|
||
|
|
---
|
||
|
|
Similar frame reduction is the simplest technique involved but greatly helps with reducing the size of animations. We iterate over the frames, compare the current frame to the first frame using [ImageChops.difference](https://pillow.readthedocs.io/en/stable/reference/ImageChops.html#PIL.ImageChops.difference) and store only the difference and it's bounding box. This way when compositing we can overlay the difference on the base image (using the bounding box to determine the offset) to create the original frame. This will be changed to be additive soon but this optimization works well at the moment.
|
||
|
|
### Tests
|
||
|
|
---
|
||
|
|
In our tests on pixel art we see around 15% gains in compression at base with further gains depending on the composition/color usage of the art
|
||
|
|
### Command line tool usage
|
||
|
|
---------------------
|
||
|
|
|
||
|
|
Using SR is rather standard, this is both a command line tool and a library, first here's the --help section generated by the binary:
|
||
|
|
|
||
|
|
|
||
|
|
|
||
|
|
```
|
||
|
|
usage: sr [-h] [-o OUTPUTPATH] [-a ANIMATION] [-f OUTPUTFORMAT]
|
||
|
|
[-i INPUTFORMAT] [-t TRANSPARENT] [-p PALETTES] [-s SCALINGFACTOR]
|
||
|
|
[-m METADATAFILE] [-e ENCRYPTIONKEY] [-x EXCLUDEALL]
|
||
|
|
inputPath
|
||
|
|
|
||
|
|
positional arguments:
|
||
|
|
inputPath Path to load input file from
|
||
|
|
|
||
|
|
options:
|
||
|
|
-h, --help show this help message and exit
|
||
|
|
-o OUTPUTPATH, --outputPath OUTPUTPATH
|
||
|
|
Output path, if this is set improperly for the output
|
||
|
|
we will infer the proper path
|
||
|
|
-a ANIMATION, --animation ANIMATION
|
||
|
|
Whether this is an animation
|
||
|
|
-f OUTPUTFORMAT, --outputFormat OUTPUTFORMAT
|
||
|
|
Format to convert to
|
||
|
|
-i INPUTFORMAT, --inputFormat INPUTFORMAT
|
||
|
|
Format to convert from
|
||
|
|
-t TRANSPARENT, --transparent TRANSPARENT
|
||
|
|
Whether output should be transparent
|
||
|
|
-p PALETTES, --palettes PALETTES
|
||
|
|
Comma seperated list of .hex palettes to create swaps
|
||
|
|
from
|
||
|
|
-s SCALINGFACTOR, --scalingFactor SCALINGFACTOR
|
||
|
|
Scale up this many times when rendering
|
||
|
|
-m METADATAFILE, --metadataFile METADATAFILE
|
||
|
|
JSON (.json) or Msgpack (.bin) encoded metadata file
|
||
|
|
-e ENCRYPTIONKEY, --encryptionKey ENCRYPTIONKEY
|
||
|
|
Encryption key for image
|
||
|
|
-x EXCLUDEALL, --excludeAll EXCLUDEALL
|
||
|
|
Only encrypt metadatas secret field
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Library
|
||
|
|
---
|
||
|
|
You can find the documentation for the library at the end of this page
|
||
|
|
### What's next?
|
||
|
|
---
|
||
|
|
Encryption, vector art, dynamic metadata and much more, check the issues to see what we're working on or add your own ideas. For now this is a weekend project though so no promises
|
||
|
|
### Acknowledgements
|
||
|
|
---
|
||
|
|
This project would not have been possible without the help of PierMesh donors, guidance by glitch artists, pixel artists and AV nerds in my childhood, and lastly but most importantly the unwavering support of my wife, partners and cat.
|
||
|
|
|
||
|
|
### Docs
|
||
|
|
---<!-- markdownlint-disable -->
|
||
|
|
|
||
|
|
# API Overview
|
||
|
|
|
||
|
|
## Modules
|
||
|
|
|
||
|
|
- [`compression`](./compression.md#module-compression)
|
||
|
|
- [`convert`](./convert.md#module-convert)
|
||
|
|
- [`sr`](./sr.md#module-sr)
|
||
|
|
- [`tests`](./tests.md#module-tests)
|
||
|
|
|
||
|
|
## Classes
|
||
|
|
|
||
|
|
- No classes
|
||
|
|
|
||
|
|
## Functions
|
||
|
|
|
||
|
|
- [`compression.indexCollapse`](./compression.md#function-indexcollapse): Collapse array with optional preset index of array, otherwise we automatically index it then return the index and collapsed array
|
||
|
|
- [`compression.rebuild`](./compression.md#function-rebuild): Rebuild collapsed array
|
||
|
|
- [`convert.fromPBytes`](./convert.md#function-frompbytes): Convert bitflip array into pixels then return the 1 dimensional list of pixels
|
||
|
|
- [`convert.getNewPixels`](./convert.md#function-getnewpixels): Compare two images and return the pixels that are different in the second image as an ImageFile and the corresponding bounding box
|
||
|
|
- [`convert.loadHexPalette`](./convert.md#function-loadhexpalette): Load hex palette from the file at the path passed in by the palette parameter, convert it to an RGB index and return the index
|
||
|
|
- [`convert.toAnimation`](./convert.md#function-toanimation): Write SRB frames to disk as either a set of files or a single animated webp or gif
|
||
|
|
- [`convert.toPBytes`](./convert.md#function-topbytes): Convert an ImageFile to bitflip array bytes, here referred to as pBytes or pixel bytes, then return the index
|
||
|
|
- [`convert.toSR`](./convert.md#function-tosr): Convert the image at imPath to an SR at outPath, optionally accepting a scaling factor for rendering and a predefined palette to use instead of auto building one
|
||
|
|
- [`convert.toSRB`](./convert.md#function-tosrb): Convert the frames at framesGlob to an SRB at outPath, with an optional scaling factor and frame offset
|
||
|
|
- [`convert.toStill`](./convert.md#function-tostill): Convert SR file to still image, optionally provide a palette for palette dependent files or for palette swapping
|
||
|
|
- [`sr.getOutputPath`](./sr.md#function-getoutputpath): Takes our argParser args, infers and returns the proper output path and what action to take (what function to use)
|
||
|
|
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
_This file was automatically generated via [lazydocs](https://github.com/ml-tooling/lazydocs)._
|
||
|
|
|
||
|
|
### LICENSE NOTICE
|
||
|
|
```
|
||
|
|
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
|
||
|
|
|
||
|
|
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
|
||
|
|
|
||
|
|
You should have received a copy of the GNU General Public License along with this program. If not, see <https://www.gnu.org/licenses/>. ```
|