Welcome to Thumbor’s documentation!¶
Whats Thumbor?¶
Thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.
It features a VERY smart detection of important points in the image for better cropping and resizing, using state-of-the-art face and feature detection algorithms (more on that in Detection Algorithms).
Using thumbor is very easy (after it is running). All you have to do is access it using an URL for an image, like this:
http://thumbor-server/unsafe/300x200/smart/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
That URL would show an image of the Big Brother Brasil participants in 300x200 using smart crop. There are several other options to the image URL configuration. You can check them in the Usage page. For more details on the /unsafe part of the URL, check the Security page.
The safe url for the above URL would look like (check Security for more details):
http://thumbor-server/K97LekICOXT9MbO3X1u8BBkrjbu5/300x200/smart/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Warning
Release 7.0.0 introduces a major breaking change due to the migration to python 3 and the modernization of our codebase. Please read the release notes for details on how to upgrade.
Contents¶
Installing¶
Installing thumbor is really easy because it supports the distutils form of packaging (http://docs.python.org/distutils/setupscript.html).
Warning
Thumbor v7.0.0 and later only supports python 3.7+. This change was important to improve our codebase and ensure it’s easier to change in future releases. More breaking changes will come, but we do not anticipate any as big as this one. Please refer to release notes for details on how to upgrade.
Stable¶
The latest stable version of thumbor is always published in the Python
Package Index (http://pypi.python.org/pypi), so it can be easily
installed using pip install thumbor or easy_install thumbor.
Ubuntu/Debian using aptitude (apt-get)¶
There’s now an officially supported ppa for thumbor if you are using aptitude.
To install using aptitude, add the following lines to your sources list:
deb http://ppa.launchpad.net/thumbor/ppa/ubuntu <your release> main
deb-src http://ppa.launchpad.net/thumbor/ppa/ubuntu <your release> main
If you are using ubuntu 12.10 (quantal), it would be:
deb http://ppa.launchpad.net/thumbor/ppa/ubuntu quantal main
deb-src http://ppa.launchpad.net/thumbor/ppa/ubuntu quantal main
Or you can add the repository to you sources list via the command line:
sudo add-apt-repository ppa:thumbor/ppa
After that just update your sources:
sudo aptitude update
And install using plain old aptitude install:
sudo aptitude install thumbor
A service will be created for you that gets started when the machine starts up (using upstart).
By default thumbor will be disabled. Open /etc/default/thumbor and
change (or remove) the flag enabled to 1 or use the command
sudo service thumbor start force=1 (force_start=1 for
thumbor<3.7.0) to temporarily start thumbor. You can also override other
defaults like the location of the configuration file by editing
/etc/default/thumbor.
The configuration for thumbor will be at /etc/thumbor.conf and the
security key at /etc/thumbor.key. There will be one instance running
at http://localhost:8888.
If you want to run many instances of thumbor you’ll need to run it in many ports. That means you’ll need to use some form of load balancing (NGINX, Apache, Varnish, Haproxy, etc).
Running many instances of thumbor is as simple as editing
/etc/default/thumbor and changing the port key to as many ports
as you want, comma-separated: port=8888,8889,8890 (for
thumbor>3.7.0).
If you need more detail head to https://launchpad.net/~thumbor/+archive/ppa.
From the source of a stable release¶
Download the latest stable source-code version here on github or Pypi and decompress it.
In the path you decompressed, execute pip install . or
python setup.py install.
From the latest version of the source¶
Clone thumbor’s repository and install it using one of the following:
pip install git+git://github.com/thumbor/thumbor.git
or
git clone git://github.com/thumbor/thumbor.git
cd thumbor
python setup.py install
Getting Started¶
If you just want to give thumbor a try, it is pretty easy to get started. It won’t take more than a minute.
Just install it with pip install thumbor and start the process with
thumbor in a console. That’s all you need to start transforming images.
The image we’ll be using in most of our examples is a Creative Commons licensed image by Snapwire:
https://github.com/thumbor/thumbor/raw/master/example.jpg
If you want to use a different image, go ahead. Any image will work for the remainder of the docs.
Note
Thumbor only understands properly encoded URIs. In order to use the URI above (or any other for that matter), we first need to encode it. This can be easily achieved by going to any modern browser’s developer console and typing:
window.encodeURIComponent(
"https://github.com/thumbor/thumbor/raw/master/example.jpg"
)
And the output will be:
https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
That’s the URL we’ll be using in our examples!
Problems installing thumbor locally¶
No worries! If you have a docker host accessible, just run:
$ docker run -p 8888:80 minimalcompact/thumbor
After downloading the image and running it, thumbor will be accessible at http://localhost:8888/.
For more information on minimalcompact/thumbor, visit their github page.
Changing its size¶
Go to your browser and enter in the url:
http://localhost:8888/unsafe/300x200/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
You should see the image with \(300px\) of width and \(200px\) of height. Just play with it in the url to see the image change.
If you just want it to be proportional to the width, enter a height of 0, like:
http://localhost:8888/unsafe/300x0/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Flipping the image¶
How about seeing it backwards? Or upside down?
Go to your browser and enter in the url:
http://localhost:8888/unsafe/-0x-0/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
You should see the waterfall backwards and upside down.
Filters¶
What if I want to change contrast or brightness?
Go to your browser and enter in the url:
http://localhost:8888/unsafe/filters:brightness(10):contrast(30)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
There are many more filters to explore. Check the Filters page for more details.
What now?¶
Ok, now that you know how amazing thumbor is, there’s actually A LOT more to it. Go check the rest of the docs to learn how to get even more from your new imaging server.
Usage¶
Using thumbor is really straightforward. thumbor offers one endpoint for retrieving the image and a very similar endpoint to retrieve metadata.
Image Endpoint¶
/hmac/trim/AxB:CxD/(adaptive-)(full-)fit-in/-Ex-F/HALIGN/VALIGN/smart/filters:FILTERNAME(ARGUMENT):FILTERNAME(ARGUMENT)/*IMAGE-URI*
- hmac is the signature that ensures Security ;
- trim removes surrounding space in images using top-left pixel color unless specified otherwise;
- AxB:CxD means manually crop the image at left-top point AxB and right-bottom point CxD;
- fit-in means that the generated image should not be auto-cropped and otherwise just fit in an imaginary box specified by ExF. If a full fit-in is specified, then the largest size is used for cropping (width instead of height, or the other way around). If adaptive fit-in is specified, it inverts requested width and height if it would get a better image definition;
- -Ex-F means resize the image to be ExF of width per height size. The minus signs mean flip horizontally and vertically;
- HALIGN is horizontal alignment of crop;
- VALIGN is vertical alignment of crop;
- smart means using smart detection of focal points;
- filters can be applied sequentially to the image before returning;
- IMAGE-URI is the encoded URI for the image you want resized.
Trim¶
Removing surrounding space in images can be done using the trim option.
Unless specified trim assumes the top-left pixel color and no tolerance (more on tolerance below).
To use it, just add a /trim part to your URL.
If you need to specify the orientation from where to get the pixel
color, just use /trim:top-left for the top-left pixel color or
/trim:bottom-right for the bottom-right pixel color.
Trim also supports color tolerance. The euclidian distance between the colors of the reference pixel and the surrounding pixels is used. If the distance is within the tolerance they’ll get trimmed. For a RGB image the tolerance would be within the range 0-442.
Manual Crop¶
The manual crop is entirely optional. This is very useful for applications that provide custom real-time cropping capabilities to their users.
The manual crop part of the url takes two points as arguments, separated by a colon. The first point is the left-top point of the cropping rectangle. The second point is the right-bottom point.
This crop is performed before the rest of the operations, so it can be used as a prepare step before resizing and smart-cropping. It is very useful when you just need to get that celebrity face on a big picture full of people, as an example.
Fit in¶
The fit-in argument specifies that the image should not be auto-cropped and auto-resized to be EXACTLY the specified size, and should be fit in an imaginary box of “E” width and “F” height, instead.
Consider an image of \(800px\) x \(600px\), and a fit of \(300px\) x \(200px\). This is how thumbor would resize it:
Consider an image of \(400px\) x \(600px\), and a fit of \(300px\) x \(200px\). This is how thumbor would resize it:
This is very useful when you need to fit an image somewhere, but you have no idea about the original image dimensions.
If a full fit-in is used, instead of using the largest size for cropping it uses the smallest one, so in the above scenarios:
For the image of \(800px\) x \(600px\), with a full fit-in of \(300px\) x \(200px\), we would get an image of \(300px\) x \(225px\).
For the image of \(400px\) x \(600px\), with a full fit-in of \(300px\) x \(200px\), we would get an image of \(300px\) x \(450px\).
Image Size¶
The image size argument specifies the size of the image that will be returned by the service. Thumbor uses smart Crop and Resize Algorithms
If you omit one of the dimensions or use zero as a value (as in \(300x\), \(300x0\), \(x200\), \(0x200\), and so on), Thumbor will determine that dimension as to be proportional to the original image. Say you have an \(800x600\) image and ask for a \(400x0\) image. Thumbor will infer that since \(400\) is half of \(800\), then the height you are looking for is half of \(600\), which is \(300px\).
If you use \(0x0\), Thumbor will use the original size of the image and thus won’t do any cropping or resizing.
If you specify one of the dimensions as the string “orig” (as in \(origx100\), \(100xorig\), \(origxorig\)), thumbor will interpret that you want that dimension to remain the same as in the original image. Consider an image of \(800x600\). If you ask for a \(300xorig\) version of it, thumbor will interpret that you want a \(300x600\) image. If you instead ask for a \(origx300\) version, thumbor will serve you an \(800x300\) image.
If you use \(origxorig\), Thumbor will use the original size of the image and thus won’t do any cropping or resizing.
The default value (in case it is omitted) for this option is to use proportional size (0) to the original image.
Horizontal Align¶
As was explained above, unless the image is of the same proportion as the desired size, some cropping will need to occur.
The horizontal align option controls where the cropping will occur if some width needs to be trimmed (unless some feature detection occurs - more on that later).
So, if we need to trim \(300px\) of the width and the current horizontal align is “left”, then we’ll trim 0px of the left of the image and \(300px\) of the right side of the image.
The possible values for this option are:
left- only trims the right side;center- trims half of the width from the left side and half from the right side;right- only trims the left side.
It is important to notice that this option is useless in case of the image being vertically trimmed, since Thumbor’s cropping algorithm only crops in one direction.
The default value (in case it is omitted) for this option is “center”.
Vertical Align¶
The vertical align option is analogous to the horizontal one, except that it controls height trimming.
So, if we need to trim \(300px\) of the height and the current vertical align is “top”, then we’ll trim \(0px\) of the top of the image and \(300px\) of the bottom side of the image.
The possible values for this option are:
top- only trims the bottom;middle- trims half of the height from the top and half from the bottom;bottom- only trims the top.
It is important to notice that this option is useless in case of the image being horizontally trimmed, since Thumbor’s cropping algorithm only crops in one direction.
The default value (in case it is omitted) for this option is “middle”.
Smart Cropping¶
Thumbor uses some very advanced techniques for obtaining important points of the image (referred to as Focal Points in the rest of this documentation).
Even though Thumbor comes with facial recognition of Focal Points as well as feature recognition, you can easily implement your own detectors as you’ll see further in the docs.
There’s not much to this option, since we’ll cover it in the Detection Algorithms page. If you use it in the url, smart cropping will be performed and will override both horizontal and vertical alignments if it finds any Focal Points.
The default value (in case it is omitted) for this option is not to use smart cropping.
Filters¶
Thumbor allows for usage of a filter pipeline that will be applied sequentially to the image. Filters are covered in the Filters page if you want to know more.
To use filters add a filters: part in your URL. Filters are like
function calls filter_name(argument, argument2, etc) and are
separated using the : character, like filters:filter_name():other_filter().
Image URI¶
This is the image URI. The format of this option depends heavily on the image loader you are using. Thumbor comes pre-packaged with an HTTP loader and a Filesystem loader.
If you use the HTTP loader, this option corresponds to the image complete URI.
If you use the Filesystem loader, this option corresponds to the path of the image from the images root.
You can learn more about the loaders in the Image loader page.
Metadata Endpoint¶
The metadata endpoint has ALL the options that the image one has, but instead of actually performing the operations in the image, it just simulates the operations.
Since it has the same options as the other endpoint, we won’t repeat all of them. To use the metadata endpoint, just add a /meta in the beginning of the url.
Say we have the following crop URL:
http://my-server.thumbor.org/unsafe/-300x-200/left/top/smart/path/to/my/nice/image.jpg
If we want the metadata on what thumbor would do, just change the url to be
http://my-server.thumbor.org/unsafe/meta/-300x-200/left/top/smart/path/to/my/nice/image.jpg
After the processing is finished, thumbor will return a json object containing metadata on the image and the operations that would have been performed.
The json looks like this:
{
thumbor: {
source: {
url: "path/to/my/nice/image.jpg",
width: 800,
height: 600
},
operations: [
{
type: "crop",
left: 10,
top: 10,
right: 300,
bottom: 200
},
{
type: "resize",
width: 300,
height: 200
},
{ type: "flip_horizontally" },
{ type: "flip_vertically" }
]
}
}
Imaging¶
Crop and Resize Algorithms¶
Note
thumbor performs the least amount of cropping possible to resize your image to the exact size you specified, without changing it’s aspect ratio.
Cropping the image¶
Before resizing the image, thumbor crops it so it has the same aspect as the desired dimensions. Let’s see an example to clarify this concept.
Consider an \(800x600\) (width x height, in pixels) image and say we want a \(400x150\) thumbnail of it. The first thing thumbor needs to do is calculate the proportion of the images:
Now that they don’t match, thumbor defines if the image needs horizontal or vertical cropping. We never crop both ways, since it’s not needed.
So, in our example to get an image of the same proportion of the target one, we need to get the picture height to be \(300px\) (using the proportional height):
Now all we need to do is cropping \(300px\) of the picture height. To determine whether to crop from the top, bottom or both we use the focal points or the horizontal alignment. If any focal points have been specified we’ll use those to find the center of mass of the image (more on that in Detection Algorithms). Otherwise we’ll use the horizontal and vertical alignments.
Let’s say that for this image no focal points were found, so we’ll use the vertical alignment to crop the height. Since we specified middle alignment for this example, we’ll crop off \(150px\) from the top and \(150px\) from the bottom of the image, similarly to this image:
Here’s an example of how thumbor would crop width or height using centered alignment:
Resizing the Image¶
Now that the image has the same proportion as the image we want, it’s just a matter of resizing it to the desired dimensions.
Flipping the Image¶
If the desired dimensions feature negative numbers, thumbor will flip them around that direction. This means that negative width specifies horizontal flip, while negative height specifies vertical flip.
Filters¶
How Filters Work¶
Thumbor handles filters in a pipeline. This means that they run sequentially in the order they are specified! Given an original image with size \(60x40\) and the following transformations:
http://localhost:8888/fit-in/100x100/filters:watermark(..):blur(..):fill(red,1):upscale()/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
The resulting image will first check if it can fit into a \(100x100\). Since it does, the filter pipeline will kick in and:
- add the watermark in the image;
- blur the whole image (including the watermark);
- Fill the outer parts of the image with red (so it will fit in \(100x100\));
- Then it will try to upscale. This will have no effect, since at this point the image is already \(100x100\).
Available Filters¶
AutoJPG¶
Usage: autojpg(enabled)
This filter overrides AUTO_PNG_TO_JPG config variable.
- enabled - Passing
True, which is the default value, you will override theAUTO_PNG_TO_JPGconfig variable and False to keep the default behavior of thus config.
http://localhost:8888/unsafe/300x300/filters:autojpg()/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Background Color¶
Usage: background_color(color)
The background_color filter sets the background layer to the specified color. This is specifically useful when converting transparent images (PNG) to JPEG
color- the color name (like in HTML) or hexadecimal rgb expression without the “#” character (see https://en.wikipedia.org/wiki/Web_colors for example). If color is “auto”, a color will be smartly chosen (based on the image pixels) to be the filling color.
The original image is:
http://localhost:8888/unsafe/fit-in/300x300/filters:background_color(blue)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fdocs%2Fimages%2Fdice_transparent_background.png
http://localhost:8888/unsafe/fit-in/300x300/filters:background_color(f00)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fdocs%2Fimages%2Fdice_transparent_background.png
http://localhost:8888/unsafe/fit-in/300x300/filters:background_color(add8e6)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fdocs%2Fimages%2Fdice_transparent_background.png
Blur¶
Usage: blur(radius [, sigma])
This filter applies a gaussian blur to the image.
radius- Radius used in the gaussian function to generate a matrix, maximum value is 150. The bigger the radius more blurred will be the image.sigma- Optional. Defaults to the same value as the radius. Sigma used in the gaussian function.
http://localhost:8888/unsafe/filters:blur(7)/http%3A%2F%2Fupload.wikimedia.org%2Fwikipedia%2Fcommons%2Fthumb%2F8%2F8a%2F2006_Ojiya_balloon_festival_011.jpg%2F159px-2006_Ojiya_balloon_festival_011.jpg
Brightness¶
Usage: brightness(amount)
This filter increases or decreases the image brightness.
amount--100 to 100- The amount (in %) to change the image brightness. Positive numbers make the image brighter and negative numbers make the image darker.
http://localhost:8888/unsafe/filters:brightness(40)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Contrast¶
Usage: contrast(amount)
This filter increases or decreases the image contrast.
amount- \(-100\) to \(100\) - The amount (in %) to change the image contrast. Positive numbers increase contrast and negative numbers decrease contrast.
http://localhost:8888/unsafe/filters:contrast(40)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/filters:contrast(-40)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Convolution¶
Usage: convolution(matrix_items, number_of_columns, should_normalize)
This filter runs a convolution matrix (or kernel) on the image. See Kernel (image processing) for details on the process. Edge pixels are always extended outside the image area.
matrix_items- Semicolon separated matrix items.number_of_columns- Number of columns in the matrix.should_normalize- Whether or not we should divide each matrix item by the sum of all items.
Normalized Matrix:
1 2 1
2 4 2
2 1 2
http://localhost:8888/unsafe/filters:convolution(1;2;1;2;4;2;1;2;1,3,true)/http://upload.wikimedia.org/wikipedia/commons/5/50/Vd-Orig.png
Matrix:
-1 -1 -1
-1 8 -1
-1 -1 -1
http://localhost:8888/unsafe/filters:convolution(-1;-1;-1;-1;8;-1;-1;-1;-1,3,false)/http://upload.wikimedia.org/wikipedia/commons/5/50/Vd-Orig.png
Cover¶
Usage: cover()
This filter is used in GIFs to extract their first frame as the image to be used as cover.
Note
This filter will only function when USE_GIFSICLE_ENGINE are set to True in thumbor.conf:
USE_GIFSICLE_ENGINE = True
No arguments.
Equalize¶
Usage: equalize()
This filter equalizes the color distribution in the image.
No arguments.
http://localhost:8888/unsafe/filters:equalize()/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Extract focal points¶
Usage: extract_focal()
When cropping, thumbor uses focal points in the image to direct the area of the image that matters most. There are several ways of finding focal points. To learn more about focal points, visit the Detection Algorithms.
In order to use the extract_focal filter, the original image must be
a thumbor URL that features manual cropping. To learn more about manual
cropping, visit the Crop and Resize Algorithms.
Using the original manual cropping points, this filter adds the cropped area (originally in the format /LEFTxTOP:RIGHTxBOTTOM/) as a focal point for the new image.
For the new image, thumbor will use as the original the image URL that was the original for the segment with the manual cropping.
This means that for an URL like:
http://localhost:8888/unsafe/300x100/filters:extract_focal()/localhost:8888/unsafe/100x150:300x200/https://upload.wikimedia.org/wikipedia/commons/thumb/2/22/Turkish_Van_Cat.jpg/546px-Turkish_Van_Cat.jpg
Thumbor will use as original the following image URL:
https://upload.wikimedia.org/wikipedia/commons/thumb/2/22/Turkish_Van_Cat.jpg/546px-Turkish_Van_Cat.jpg
Original Image:
Cat’s eye cropped:
http://localhost:8888/unsafe/100x150:300x200/https://upload.wikimedia.org/wikipedia/commons/thumb/2/22/Turkish_Van_Cat.jpg/546px-Turkish_Van_Cat.jpg
A bigger image based on above’s crop with the extract_focal() filter:
http://localhost:8888/unsafe/300x100/filters:extract_focal()/localhost:8888/unsafe/100x150:300x200/https://upload.wikimedia.org/wikipedia/commons/thumb/2/22/Turkish_Van_Cat.jpg/546px-Turkish_Van_Cat.jpg
Without the filter that would be the result:
http://localhost:8888/unsafe/300x100/localhost:8888/unsafe/100x150:300x200/https://upload.wikimedia.org/wikipedia/commons/thumb/2/22/Turkish_Van_Cat.jpg/546px-Turkish_Van_Cat.jpg
Filling¶
Usage: fill(color[,fill_transparent])
This filter returns an image sized exactly as requested independently of its ratio. It will fill the missing area with the specified color. It is usually combined with the “fit-in” or “adaptive-fit-in” options.
color- the color name (like in HTML) or hexadecimal RGB expression without the “#” character (see https://en.wikipedia.org/wiki/Web_colors for example).If color is “transparent” and the image format, supports transparency the filling color is transparent.
Warning
Some engines (like OpenCV engine) do not support transparency.
If color is “auto”, a color is smartly chosen (based on the image pixels) as the filling color.
If color is “blur”, the missing parts are filled with blurred original image.
fill_transparent- a boolean to specify whether transparent areas of the image should be filled or not. Accepted values are either true, false, 1 or 0. This argument is optional and the default value is false.
The original image is:
http://localhost:8888/unsafe/fit-in/300x300/filters:fill(blue)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/fit-in/300x300/filters:fill(f00)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/fit-in/300x300/filters:fill(add8e6)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/fit-in/300x300/filters:fill(auto)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/fit-in/300x300/filters:fill(blur)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
The original image is:
http://localhost:8888/unsafe/fit-in/300x225/filters:fill(blue,1)/https://github.com/thumbor/thumbor/wiki/dice_transparent_background.png
http://localhost:8888/unsafe/fit-in/300x225/filters:fill(f00,true)/https://github.com/thumbor/thumbor/wiki/dice_transparent_background.png
http://localhost:8888/unsafe/fit-in/300x225/filters:fill(add8e6,1)/https://github.com/thumbor/thumbor/wiki/dice_transparent_background.png
http://localhost:8888/unsafe/fit-in/300x225/filters:fill(auto,true)/https://github.com/thumbor/thumbor/wiki/dice_transparent_background.png
http://localhost:8888/unsafe/fit-in/300x225/filters:fill(blur,true)/https://github.com/thumbor/thumbor/wiki/dice_transparent_background.png
Focal¶
Usage: focal(<left>x<top>:<right>x<bottom>)
This filter adds a focal point, which is used in later transforms.
left, top, right, bottom: All mandatory arguments in the<left>x<top>:<right>x<bottom>format.
Before cropping with specific focal point:
http://localhost:8888/unsafe/400x100/filters:focal(146x206:279x360)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
After specifying the focal point:
Warning
When using this filter together with detectors, extract focal points filter or metadata parameter, unexpected behavior may occur.
Format¶
Usage: format(image-format)
This filter specifies the output format of the image. The output must be one of: “webp”, “jpeg”, “gif”, “png”, or “avif”.
image-format- The output format of the resulting image.
http://localhost:8888/unsafe/filters:format(webp)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Grayscale¶
Usage: grayscale()
This filter changes the image to grayscale.
No arguments.
http://localhost:8888/unsafe/filters:grayscale()/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Max bytes¶
Usage: max_bytes(number-of-bytes)
This filter automatically degrades the quality of the image until the image is under the specified amount of bytes.
number-of-bytes- The maximum number of bytes for the given image.
Compressing the original image to less than 7.5k (ended up with ~7kb):
http://localhost:8888/unsafe/filters:max_bytes(7500)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
No upscale¶
Usage: no_upscale()
This filter tells thumbor not to upscale your images.
This means that if an original image is \(300px\) width by \(200px\) height and you ask for a \(600x400\) image, thumbor will still return a \(300x200\) image.
No arguments allowed.
http://localhost:8888/unsafe/filters:no_upscale()/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Noise¶
Usage: noise(amount)
This filter adds noise to the image.
amount-0% to 100%- The amount of noise to add to the image.
http://localhost:8888/unsafe/filters:noise(40)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Proportion¶
Usage: proportion(percentage)
This filter applies the specified porportion to the image’s height and width when cropping.
percentage- The float percentage of the proportion (0.0 to 1.0).
http://localhost:8888/unsafe/filters:proportion(0.5)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Quality¶
Usage: quality(amount)
This filter changes the overall quality of the JPEG image (does nothing for PNGs or GIFs).
http://localhost:8888/unsafe/filters:quality(40)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Red eye¶
Not documented yet
RGB¶
Usage: rgb(rAmount, gAmount, bAmount)
This filter changes the amount of color in each of the three channels.
rAmount- The amount of redness in the picture. Can range from -100 to 100 in percentage.gAmount- The amount of greenness in the picture. Can range from -100 to 100 in percentage.bAmount- The amount of blueness in the picture. Can range from -100 to 100 in percentage.
http://localhost:8888/unsafe/filters:rgb(20,-20,40)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Rotate¶
Usage: rotate(angle)
This filter rotates the given image according to the angle value passed.
Note
This filter rotates the image according to the engine. For the PIL engine the rotation is done counter-clockwise.
angle-0 to 359- The euler angle to rotate the image by. Numbers greater or equal than 360 will be transformed to a equivalent angle between 0 and 359.
http://localhost:8888/unsafe/filters:rotate(90)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Round corners¶
Usage: round_corner(a|b,r,g,b,[transparent])
This filter adds rounded corners to the image using the specified color as background.
a|b- amount of pixels to use as radius. The argumentbis not required, but it specifies the second value for the ellipsis used for the radius.transparent- Optional. If set to true/1, the background will be transparent.
http://localhost:8888/unsafe/filters:round_corner(20,255,255,255)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/filters:round_corner(20|40,0,0,0)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
http://localhost:8888/unsafe/filters:round_corner(30,0,0,0,1)/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Saturation¶
Usage: saturation(amount)
This filter increases or decreases the image saturation.
amount- \(-100\) to \(100\) - The amount (in %) to change the image saturation. Positive numbers increase saturation and negative numbers decrease saturation.
/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg){kind=link}
/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg){kind=link}
Sharpen¶
Usage: sharpen(sharpen_amount,sharpen_radius,luminance_only)
This filter enhances apparent sharpness of the image. It’s heavily based on Marco Rossini’s excellent Wavelet sharpen GIMP plugin. Check http://registry.gimp.org/node/9836 for details about how it work.
sharpen_amount- Sharpen amount. Typical values are between \(0.0\) and \(10.0\).sharpen_radius- Sharpen radius. Typical values are between \(0.0\) and \(2.0\).luminance_only- Sharpen only luminance channel. Values can betrueorfalse.
http://localhost:8888/unsafe/filters:sharpen(2,1.0,true)/http://videoprocessing.ucsd.edu/~stanleychan/research/pix/Blurred_foreman_0005.png
http://localhost:8888/unsafe/filters:sharpen(1.5,0.5,true)/http://images.cambridgeincolour.com/tutorials/sharpening_eagle2-original.jpg
Stretch¶
Usage: stretch()
This filter stretches the image until it fits the required width and height, instead of cropping the image.
http://localhost:8888/unsafe/200x100/filters:stretch()/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Strip EXIF¶
Usage: strip_exif()
This filter removes any Exif information in the resulting image.
This is useful if you have set the configuration PRESERVE_EXIF_INFO = True but still wish to overwrite this behavior in some cases
(e.g. for image icons)
No arguments
http://localhost:8888/unsafe/filters:strip\_exif()/http://www.arte.tv/static-epgapi/057460-011-A.jpg
Strip ICC¶
Usage: strip_icc()
This filter removes any ICC information in the resulting image. Even though the image might be smaller, removing ICC information may result in loss of quality.
No arguments
http://localhost:8888/unsafe/filters:strip\_icc()/http://videoprocessing.ucsd.edu/~stanleychan/research/pix/Blurred_foreman_0005.png
Upscale¶
Usage: upscale()
This filter tells thumbor to upscale your images. This only makes sense with “fit-in” or “adaptive-fit-in”.
This means that if an original image is \(300px\) width by \(200px\) height and you ask for a \(600x500\) image, the filter will resize it to \(600x400\).
No arguments allowed.
http://localhost:8888/unsafe/fit-in/600x500/filters:upscale()/https://raw.githubusercontent.com/thumbor/thumbor/e86324e49d7e53acc2a8057e43f3fdd2ca5cea75/docs/images/dice_transparent_background.png
Watermark¶
Usage: watermark(imageUrl, x, y, alpha [, w_ratio [, h_ratio]])
This filter adds a watermark to the image. It can be positioned inside the image with the alpha channel specified and optionally resized based on the image size by specifying the ratio (see Resizing).
imageUrl- Watermark image URL. It is very important to understand that the same image loader that Thumbor uses will be used here. If this URL contains parentheses they MUST be url encoded, since these are the characters Thumbor uses as delimiters for filter parameters.x- Horizontal position that the watermark will be in. Positive numbers indicate position from the left and negative numbers indicate position from the right. If the value is ‘center’ (without the single quotes), the watermark will be centered horizontally. If the value is ‘repeat’ (without the single quotes), the watermark will be repeated horizontally. If the value is a positive or negative number followed by a ‘p’ (ex. 20p) it will calculate the value from the image width as percentagey- Vertical position that the watermark will be in. Positive numbers indicate position from the top and negative numbers indicate position from the bottom. If the value is ‘center’ (without the single quotes), the watermark will be centered vertically. If the value is ‘repeat’ (without the single quotes), the watermark will be repeated vertically If the value is a positive or negative number followed by a ‘p’ (ex. 20p) it will calculate the value from the image height as percentagealpha- Watermark image transparency. Should be a number between 0 (fully opaque) and 100 (fully transparent).w_ratio- percentage of the width of the image the watermark should fit-in, defaults to ‘none’ (without the single quotes) which means it won’t be limited in the width on resizing but also won’t be resized based on this valueh_ratio- percentage of the height of the image the watermark should fit-in, defaults to ‘none’ (without the single quotes) which means it won’t be limited in the height on resizing but also won’t be resized based on this value
http://thumbor-server/filters:watermark(http://my.site.com/img.png,-10,-10,50)/some/image.jpg
http://thumbor-server/filters:watermark(http://my.site.com/img.png,10p,-20p,50)/some/image.jpg
Resizing is being done by defining borders the watermark needs to fit in or being upscaled to. The ratio of the watermark will not be changed and will be expanded or shrinked to the size which fits best into the borders.
Some examples are shown below with an original image having width=300 and height=200 and an imaginary watermark having width=30 and height=40. Borders are shown in red and the watermark drafted in green.
Considering original image to be 300x200:
watermark(imageUrl, 30, 10, 50, 20)
20% of the width: 300px*0.2 = 60px so the original watermark width is 30px which means it can be resized by 2.
Because the height isn’t limited it can grow to 2x40px which is 80px.
watermark(imageUrl, 30, 10, 50, none, 15)
15% of the height: 200px*0.15 = 30px so the original watermark height is 40px which means it has to shrink by 25%.
Because the width isn’t limited it can shrink to 0.75*30px which is 22.5px (rounded to 23px).
watermark(imageUrl, 30, 10, 50, 30, 30)
30% of the width: 300px*0.3 = 90px
and
30% of the height: 200px*0.3 = 60px
so the original watermark width is 30px but cannot use 90px because then (to keep the ratio) the height would need to become (40/30)*90px=120px but only 60px is allowed.
Therefor the height is limiting the resizing here and height would become 60px and width would be (30/40)*60px=45px which fits into the 90px border.
Detectors¶
Enabling detectors¶
Out of the box, thumbor does not enable any feature or facial detection. Enabling it is pretty easy, though.
Note
Starting with release 7.0.0 thumbor depends on opencv-python-headless. This means that it should be extremely easy to use the face and feature detectors.
For information on all built-in detectors check the Available detectors page.
Configuration¶
In order to tell thumbor what detectors it should run in the original
image, you must add them to your thumbor.conf file in the following
key:
DETECTORS = [
'thumbor.detectors.face_detector',
'thumbor.detectors.feature_detector',
]
The above configuration tells thumbor that it should run both the facial detection and the feature detection. These are mutually exclusive, meaning that if a face is detected, the feature detector won’t be run.
Using it¶
After restarting thumbor, it should be as easy as adding a /smart
option to your URLs, like:
http://localhost:8888/unsafe/200x400/smart/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Note
Whenever you are not sure what thumbor is “seeing”, use the debug mode:
http://localhost:8888/unsafe/debug/200x400/smart/https%3A%2F%2Fgithub.com%2Fthumbor%2Fthumbor%2Fraw%2Fmaster%2Fexample.jpg
Thumbor will draw a square on all focal points it found. That way you can be sure of why an image was cropped the way it was.
Lazy Detection¶
Facial detection can be pretty expensive for thumbor, so it is not advisable to do it synchronously. Please refer to the Lazy Detection page for instructions on using it.
Available Detectors¶
A list of available detectors can be found at Available detectors.
Detection Algorithms¶
If the smart mode of thumbor has been specified in the uri (by the /smart portion of it), thumbor will use it’s smart detectors to find focal points.
thumbor comes pre-packaged with two focal-point detection algorithms: facial and feature. First it tries to identify faces and if it can’t find any, it tries to identify features (more on that below).
Facial Detection¶
For instructions on how to get facial detection coordinates see Metadata Endpoint .
Note
thumbor uses OpenCV (http://opencv.org) to detect faces. OpenCV returns the rectangle coordinates for the faces it identifies. You can specify the HAAR file Thumbor should use for identification.
Notice how red rectangles show the areas identified as faces:
After retrieving these squares from OpenCV, thumbor calculates the center of mass of the image using weighted average.
Consider that OpenCV returned 3 squares at (10, 10, 100, 100), (150, 100, 100, 100), (300, 300, 80, 50), being (x, y, width, height), as such:
In order to find the center of mass for all the faces, we must first find the center and weight of each rectangle. We define weight in this scenario as the area of the rectangle.
So, for the faces in our example (*x*,*y* being the coordinates of the rectangle’s center and *z* the rectangle weight):
Face 1:
- \(x = (10 + 100) ÷ 2 = 55\)
- \(y = (10 + 100) ÷ 2 = 55\)
- \(z = 100 * 100 = 10000\)
Face 2:
- \(x = (150 + 100) ÷ 2 = 125\)
- \(y = (100 + 100) ÷ 2 = 100\)
- \(z = 100 * 100 = 10000\)
Face 3:
- \(x = (300 + 80) ÷ 2 = 190\)
- \(y = (300 + 50) ÷ 2 = 175\)
- \(z = 80 * 50 = 4000\)
In order to find the center of mass we’ll do a weighted average of the X and Y coordinates of the faces using:
Horizontal Axis - X: \(((55 * 10000) + (125 * 10000) + (190 * 4000)) ÷ 24000 = 106\)
Vertical Axis - Y: \(((55 * 10000) + (100 * 10000) + (175 * 4000)) ÷ 24000 = 93\)
So for the faces found by OpenCV in that image we have the center of mass of the picture being \(106x93\).
After finding the center of mass we can use it as the focal point for cropping. Given an image of \(800x600\) and a focal point at \(106x93\), we need to determine the percentage that needs to be cropped from the top, bottom, left and right sides of the image.
To determine the percentage we use simple math to figure how far from the top and the left side the center of mass is:
From the left - \(left = 106 ÷ 800 * 100 = 13.25\%\)
From the top - \(top = 93 ÷ 600 * 100 = 15.50\%\)
Using the same example from the Crop and Resize Algorithms page, we need to crop \(300px\) out of the height of the image. In possession of the percentages of crop above, we can calculate how much we need to crop out of the top and bottom with:
Top - \(top = 300 * 0.155 ~= 46\)
Bottom - just subtract top \(46px\) from the amount of crop (300px): \(bottom = 300 - 46 = 254\)
So, now we now we have to remove \(46px\) out of the top of the picture and \(254px\) out of the bottom of the picture. In an \(800x600\) picture, that means cropping from (0, 46) to (800, 346), resulting in an \(800x300px\) image.
Assuming we would crop \(300px\) horizontally, the cropping would be:
Left - \(left = 300 * 0.135 ~= 40\)
Right - just subtract left \(40px\) from the amount of crop
(300px): \(right = 300 - 40 = 260\)
In an image of \(800x600\), that means cropping from (40, 0) to (540, 600), resulting in a \(500x600px\) image. This would not be the case for this image, though.
Feature Detection¶
If no faces are found in the picture, we still try to find important features in the image, provided by the Good Features to Track Algorithm in OpenCV (http://bit.ly/evAU95).
According to OpenCV documentation, this algorithm finds “important” corners in the image. It then returns a list of (x, y) values.
We can see the detection taking place in the following images:
The points identified by the good features algorithm:
The cropping based in these features is analogous to the face one, except that all points have a weight of *1.0* and are already their centers.
Let’s consider that we found 3 feature points: 10x15, 30x40, 25x60. To find the center of mass we would do ((10 + 30 + 25) / 3 ~= 22) to find the horizontal component and ((15 + 40 + 60) / 3 ~= 39) for the vertical one. This means that our center of mass in this scenario is *22x39*.
Given an image of 800x600 and a center of mass of 22x39, let’s find the left and top percentages:
From the left - \(22 / 800 * 100 = 2.75\%\)
From the top - \(93 / 600 * 100 = 6.50\%\)
Assuming we are cropping 300px of the height, we’ll crop top and bottom according to:
Top - \(300 * 0.0275 ~= 9\)
Bottom - just subtract top (9px) from the amount of crop (300px) - \(300 - 9 = 291\)
In an image of 800x600, that means cropping from (0, 9) to (800, 309), resulting in a 800x300px image.
If we were cropping 300px of the width instead, we would crop left and right according to:
Left - \(300 * 0.065 ~= 20\)
Right - just subtract left (20px) from the amount of crop (300px) - \(300 - 20 = 280\)
In an image of 800x600, that means cropping from (20, 0) to (520, 600), resulting in a 500x600px image.
Available detectors¶
Face Detector¶
thumbor.detectors.face_detector
It detects faces, It considers the frontal part of the face for detection.
Feature Detector¶
thumbor.detectors.feature_detector
Detector used to find relevant focal points in the image. “Features” in this case and in other cases such as machine learning, are pieces of information (in this case, pieces of the image) that are relevant to solving a computational problem. For Thumbor we use this set of focal points to identify faces, for example. We use the first 10 set of points found.
Profile Detector¶
thumbor.detectors.profile_detector
It detects faces, It considers the side part of the face for detection.
Queued Detector¶
thumbor.detectors.queued_detector
Detector used to allow face detection process asynchronously.
Lazy Detection¶
Rationale¶
Thumbor performs pipeline detection of focal points for a given image. What this means is that it tries to determine one detection at a time, only skipping to the next if the current one fails.
We could configure it to run frontal face detection, then if it fails, try profile face detection and if it fails, best features detection.
As you can imagine, this is a cumbersome process and can take up precious cpu time from your server(s), eventually leading it to starvation of CPU. This is why we’ve implemented what we call Queued Detection.
Queued Detection¶
Configuring thumbor for lazy detecting is as simple as specifying a detector that supports queued detection.
Thumbor ships with three such detectors, called:
thumbor.detectors.queued_detector.queued_complete_detectorthumbor.detectors.queued_detector.queued_face_detectorthumbor.detectors.queued_detector.queued_feature_detector
These are responsible, respectively, for pipeline detection of face and feature, only face or only feature.
You can check what additional configuration you need to add to your configuration file (thumbor.conf) in order to have the bundled detectors working.
How thumbor deals with queued detection?¶
When an image request arrives with a flag of “smart” detection, a call is made to the queued detector and it tells thumbor to skip smart detection and to serve the image with non-smart cropping (much faster).
The call to the queued detector places a message in a Redis Queue that will later be processed in order to detect focal points in the image.
The next time a request arrive for the same image and with a flag of “smart” detection, if information on detection is already available (if the message in the queue has already been processed), thumbor uses that info to do smart cropping and serves the result.
If the image still hasn’t been processed, the same process from before applies, except thumbor won’t place another message in the queue. This is intended as a way not to flood the queue with requests for the same image.
Redis Support¶
Thumbor supports Redis single node. and Redis sentinel.
Image loader¶
Pre-packaged loaders¶
thumbor comes pre-packaged with http and filesystem loaders.
Http loader¶
The http loader gets the original image portion of the URI and performs an HTTP GET to it. It then returns the image’s string representation.
The http loader uses the ALLOWED_SOURCES configuration to determine whether or not an image is from a trusted source and can thus be loaded.
You can specify the maximum size of the source image to be loaded. The http loader first gets the image size (without loading its contents), checks against your specified size and returns 404 if the source image size is larger than the max size. The max size option is MAX_SOURCE_SIZE and the default is no maximum size.
To use it you should set the LOADER configuration to ‘thumbor.loaders.http_loader’.
Https loader¶
The https loader works the same way as the http loader, except that it defaults to https instead of http.
To use it you should set the LOADER configuration to ‘thumbor.loaders.https_loader’.
Strict https loader¶
The strict https loader works the same way as the http loader, except that it only allows to load images over https.
To use it you should set the LOADER configuration to ‘thumbor.loaders.strict_https_loader’.
File loader¶
The file loader gets the original image portion of the URI and retrieves the file from the file system from a known path specified by the FILE_LOADER_ROOT_PATH configuration.
It joins the specified path with the configured root path and reads the image file if it exists.
To use it you should set the LOADER configuration to ‘thumbor.loaders.file_loader’.
File loader with http loader fallback¶
In some environments you need both kinds of file loading. For this use case you can use as loader with built-in fallback.
This loader will try to load images from local file storage. In case of an error the loader retry to load image with http_loader. If both attempts failed you’ll get an error.
To use it you should set the LOADER configuration to ‘thumbor.loaders.file_loader_http_fallback’.
Compatibility Loader¶
The compatibility loader allows you to use legacy loaders (that do not support AsyncIO) in order to make it easier to transition to thumbor’s Python 3 version.
To use it you should set the LOADER configuration to ‘thumbor.compatibility.loader’.
You also need to specify what’s the legacy loader that the compatibility loader will use. Just set the COMPATIBILITY_LEGACY_LOADER configuration to the full name of the legacy loader you want to use. i.e.: COMPATIBILITY_LEGACY_LOADER = ‘tc_aws.loaders.s3_loader’
Image storage¶
thumbor uses image storages to perform less retrievals of images from the sources, thus potentially saving expensive resources (such as outbound network).
Pre-Packaged Storages¶
thumbor comes with filesystem and a mixed storage. There’s also a nostorage storage for debugging or benchmarking purposes.
Filesystem Storage¶
thumbor can store original images in the filesystem.
The file storage uses the FILE_STORAGE_ROOT_PATH configuration to save the images. It then joins the original image part of the URI to create the proper path to store the image in the filesystem.
There’s a STORAGE_EXPIRATION_SECONDS option that will determine the time in seconds that a file is considered to be expired. When a file is expired, thumbor will try to retrieve the file using the specified Image loader.
To use the filesystem storage set the configuration option of STORAGE to ‘thumbor.storages.file_storage’.
NoStorage Storage¶
This is a storage intended for debugging or benchmarking purposes. It does not store any images and always returns None when thumbor asks for an image.
In order to use this storage set the configuration option of STORAGE to ‘thumbor.storages.no_storage’.
MixedStorage Storage¶
This is a storage intended for scenarios where you want to store the original images files one way and the security key another (or detector information).
A good example would be storing files in the filesystem, while storing security keys in a database.
In order to use this storage set the configuration option of STORAGE to ‘thumbor.storages.mixed_storage’.
You must specify the MIXED_STORAGE_FILE_STORAGE,
MIXED_STORAGE_CRYPTO_STORAGE and MIXED_STORAGE_DETECTOR_STORAGE
options to define the original images storage, the security key storage
and the detector results storage, respectively. Here’s a sample
configuration:
MIXED_STORAGE_FILE_STORAGE = 'thumbor.storages.file_storage'
MIXED_STORAGE_CRYPTO_STORAGE = 'thumbor.storages.redis_storage'
MIXED_STORAGE_DETECTOR_STORAGE = 'thumbor.storages.redis_storage'
FILE_STORAGE_ROOT_PATH = '/tmp/mypath'
REDIS_STORAGE_SERVER_HOST = 'localhost'
REDIS_STORAGE_SERVER_PORT = 6379
REDIS_STORAGE_SERVER_DB = 0
As you can see, you still have to tell thumbor the specific configurations for each storage you choose.
Compatibility Storage¶
The compatibility storage allows you to use legacy storages (that do not support AsyncIO) in order to make it easier to transition to thumbor’s Python 3 version.
To use it you should set the STORAGE configuration to ‘thumbor.compatibility.storage’.
You also need to specify what’s the legacy storage that the compatibility storage will use. Just set the COMPATIBILITY_LEGACY_STORAGE configuration to the full name of the legacy storage you want to use. i.e.: COMPATIBILITY_LEGACY_STORAGE = ‘tc_aws.storages.s3_storage’
Result Storage¶
thumbor uses a result storage to improve the speed of responding subsequent requests for the same image.
When a request for a given image with a set of parameters arrive, thumbor processes the request and before returning it, asks for the result storage to store it.
The next time the same request arrives, it will get it from the result storage and return it, thus saving a lot of processing.
Pre-packaged result storages¶
thumbor comes pre-packaged with a filesystem result storage.
Filesystem¶
The file system result storage, as the name implies, stores images in the filesystem.
Images are stored in whatever path is specified in the
RESULT_STORAGE_FILE_STORAGE_ROOT_PATH, and consequently retrieved
from the same path.
By default, the file system result storage keeps images forever. You are
allowed to specify an expiration, though, using the
RESULT_STORAGE_EXPIRATION_SECONDS configuration. Again, as the name
implies, it specifies the number of seconds with which files expire.
To use it you should set the RESULT_STORAGE configuration to
'thumbor.result_storages.file_storage'.
Compatibility Result Storage¶
The compatibility result storage allows you to use legacy result storages (that do not support AsyncIO) in order to make it easier to transition to thumbor’s Python 3 version.
To use it you should set the RESULT_STORAGE configuration to ‘thumbor.compatibility.result_storage’.
You also need to specify what’s the legacy result storage that the compatibility result storage will use. Just set the COMPATIBILITY_LEGACY_RESULT_STORAGE configuration to the full name of the legacy result storage you want to use. i.e.: COMPATIBILITY_LEGACY_RESULT_STORAGE = ‘tc_aws.result_storages.s3_storage’
Optimizers¶
Optimizers are utilities that will fine-tune some aspect of the result image thumbor generates.
Even though optimizers can change images in any way, the usual use cases for these are:
- Reduce image weight in bytes;
- Improve image quality.
Built-in Optimizers¶
jpegtran¶
Jpegtran is a lossless jpeg optimizer which can make your jpegs smaller by optimizing DCT coefficients. Information on jpegtran can be a bit difficult to find but the linux man page is pretty good: https://linux.die.net/man/1/jpegtran
Jpegtran can be used in conjunction with Thumbor. If the optimizer has been activated, Thumbor will first process your jpeg normally then it will hand the jpeg off to jpegtran for further optimizations before Thumbor returns the final image.
To use jpegtran with Thumbor you must first install jpegtran, various linux distros often provide a package by the same name or it can be installed from source. You should make sure that jpegtran is in PATH, do a which jpegtran and you should see an absolute path where the jpegtran resides. It is also possible to use mozjpeg’s version of jpegtran as a drop-in replacement of libjpeg-turbo’s version.
You also need to enable the Thumbor jpegtran optimizer in your thumbor.conf, like so:
OPTIMIZERS = [
'thumbor.optimizers.jpegtran'
]
You can also manually specify the jpegtran path, like this:
JPEGTRAN_PATH=/usr/local/bin/jpegtran
Once activated, no extra url parameters are needed - jpegtran will run on all jpegs automatically. If you have opted to use progressive jpegs via the PROGRESSIVE_JPEG option, jpegtran will also honor and product progressive jpegs.
It is possible to supply progressive scans file via JPEGTRAN_SCANS_FILE config option.
gifv¶
The gifv optimizer is able to convert gifs to mp4 or webm videos, often resulting in dramatically smaller sized files.
Gifv is categorized as experimental and should be used with caution. It uses ffmpeg to convert gifs to videos and so it’s sensitive to changes with ffmpeg. It’s recommended to lock your ffmpeg version with a fixed version (chef, docker, etc) and if updating make sure to check that the update doesn’t break gifv. FFmpeg version 3.2.4 is the current recommended version. Later version, such as 3.3 will break the proper conversion of gif delays to frame durations in videos … meaning videos will not be the same length as equivelant gifs.
To enable gifv, ensure ffmpeg is in PATH and enable the optimizer in your config:
OPTIMIZERS = [
'thumbor.optimizers.gifv',
]
Once activated, you must add the gifv() option to your filters list. An example request might look like this:
http://localhost:8888/unsafe/filters:gifv()/http://localhost/livingroom.gif
The above example will default to using the mp4 video container with h264 video. You can also be explicit:
http://localhost:8888/unsafe/filters:gifv(mp4)/http://localhost/livingroom.gif
or use explicitly specify webm
http://localhost:8888/unsafe/filters:gifv(webm)/http://localhost/livingroom.gif
Because videos (in mp4 or webm format) cannot contain alpha transparency a background color will be automatically added. The default color is white. You can also specify a background color:
http://localhost:8888/unsafe/filters:gifv():background_color(ff00ff)/http://localhost/livingroom.gif
http://localhost:8888/unsafe/filters:gifv():background_color(f0f)/http://localhost/livingroom.gif
http://localhost:8888/unsafe/filters:gifv():background_color(magenta)/http://localhost/livingroom.gif
The color must be specified in 6 character hex, 3 character hex or color name. But 6 or 3 character hex are the preferred formats. Including a # symbol in your color will break the url if not url encoded and thumbor will error on the request. The recommendation is to not use them at all which also makes urls shorter. But if you must a leading %23 will probably work.
Customizing Thumbor¶
Custom Storages¶
If the built-in storages do not suit your needs, you can always implement your own storage and use it in the STORAGE configuration.
All you have to do is create a class called Storage that inherits from BaseStorage in your module, as can be seen in https://github.com/thumbor/thumbor/blob/master/thumbor/storages/file_storage.py.
Custom Image Loaders¶
If thumbor image loaders do not meet your needs you can implement a new image loader.
The structure of the module you should implement can be seen in the http loader at https://github.com/thumbor/thumbor/blob/master/thumbor/loaders/http_loader.py.
The only required method to implement is the one that receives the portion of the URI that has the original image path, named load. This method also receives a callback and should call the callback with the results of reading the image.
Another example can be seen in the filesystem loader at https://github.com/thumbor/thumbor/blob/master/thumbor/loaders/file_loader.py.
You can optionally implement a validate(URI) method that thumbor will call to make sure that your loader can accept the user required URI.
Custom Result Storages¶
In order to implement your own result storage, you have to implement a few methods. A reference implementation can be found at the File Storage.
The required methods are put, get, validate_path and
normalize_path.
Custom Filters¶
Filters are an easy way to transform images using a pipeline. Creating a new filter is very simple, as we’ll see.
The first step is creating a filter class that inherits from thumbor.filters.BaseFilter and naming it Filter:
from thumbor.filters import BaseFilter
class Filter(BaseFilter):
pass
The next step is actually implementing the filter. Let’s say we want to create
the quality(99) filter, a filter that takes a number parameter and
sets the image quality to that parameter.
Note
Yep, this filter already exists and is built-in, but it is simple enough that we can talk on how to do it. Let’s get on with it.
from thumbor.filters import BaseFilter
class Filter(BaseFilter):
@filter_method(BaseFilter.PositiveNumber)
async def quality(self, value):
self.context.request.quality = value
Let’s analyse it:
- The
filter_methoddecorator takes as parameters any number of types (more on types below) you want to have as arguments to your filter; - The filter method should be named according to how you want it to be invoked by thumbor (a.k.a the URL part). In our example, our filter will be invoked with
quality(99); - The filter method is just an async function that you can do whatever you need with the image.
And that’s it, we got our filter. In order to use it, we need to put it in our thumbor.conf:
from thumbor.filters import BUILTIN_FILTERS
FILTERS = BUILTIN_FILTERS + [
'mylib.filters.quality',
]
Available Filter Argument Types¶
Each parameter type has a regular expression that matches arguments of the given type, as well as a python type.
For more details on each of the types, check BaseFilter class in thumbor’s codebase.
BaseFilter.PositiveNumber;BaseFilter.PositiveNonZeroNumber;BaseFilter.NegativeNumber;BaseFilter.Number;DecimalNumber;Boolean;String.
Custom Engines¶
TBW.
Custom detection¶
If you need more detection than the pre-packaged detectors are able to give you (i.e.: you need to detect glasses), you can always implement your own detectors.
If your detector can be found using python’s import mechanism, thumbor will be able to use it. Just add its full name to the detectors Configuration.
Creating a Custom Detector¶
As you can see here https://github.com/thumbor/thumbor/blob/master/thumbor/detectors/face_detector/__init__.py it is pretty easy to implement your own custom detector.
All you have to do is create a class that inherits from BaseDetector and implement a detect method that receives a context dictionary.
In the context dictionary there’s a key called “focal_points” to which you should append any focal points you found in the picture (using the FocalPoint class).
If your detector does not find any points, simple call the next() method passing in the context, so further detection can occur.
Custom Image Optimizers¶
TBW.
Custom Error Handlers¶
Writing your own error handler is very simple. Just create a class
called ErrorHandler, like the one below:
# Class that lives in mylib.error_handling
class ErrorHandler:
def __init__(self, config):
# perform any initialization needed
pass
def handle_error(self, context, handler, exception):
# do your thing here
# context is thumbor's context for the current request
# handler is tornado's request handler for the current request
# exception is the error that occurred
When you have your handler done, just put it’s full name in thumbor.conf
and make sure thumbor can import it (it’s somewhere in PYTHONPATH).
You also need to set USE_CUSTOM_ERROR_HANDLING to True.
USE_CUSTOM_ERROR_HANDLING = True
ERROR_HANDLER_MODULE = 'mylib.error_handling'
Custom Handler Lists¶
Handler Lists are responsible for adding new handlers to thumbor.
Even thumbor’s own handlers (other than the default image crop handler) are added using handler lists(healthcheck, blacklist…).
Built-in Handler Lists¶
Thumbor comes with three handler lists built-in:
thumbor.handler_lists.healthcheck;thumbor.handler_lists.blacklist;thumbor.handler_lists.upload.
The healthcheck handler list adds a handler at whatever is in the HEALTHCHECK_ROUTE config.
The blacklist handler list adds a /blacklist handler that can be used to blacklist images.
The upload handler list adds two handlers for uploading and retrieving uploaded images.
Writing a new Handler List¶
Creating your own handler list is as simple as creating a new module with a get_handlers method:
from typing import Any, cast
from thumbor.handler_lists import HandlerList
from my.handlers.index import IndexHandler
def get_handlers(context: Any) -> HandlerList:
something_enabled = cast(bool, self.context.config.SOMETHING_ENABLED)
if not something_enabled:
return []
return [
(r"/my-url/?", IndexHandler, {"context": self.context}),
]
After your handler list can be imported with python (check with python -c 'import <<your handler list module>>'),
just add it to thumbor’s config:
from thumbor.handler_lists import BUILTIN_HANDLERS
# Two things worth noticing here:
# 1) The handler list order indicates precedence, so whatever matches first will be executed;
# 2) Please do not forget thumbor's built-ins or you'll kill thumbor functionality.
HANDLER_LISTS = BUILTIN_HANDLERS + [
"my.handler_list',
]
Plugins¶
With its pluggable architecture, thumbor provides extension points for a myriad of plug-in: storages, loaders, detectors, filters.
If your plug-in is not listed here, please create an issue with the details and we’ll add it here.
Storages¶
thumbor_aws (by Thumbor Community)¶
Thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.
AWS is a cloud service, providing - among other things - storage capabilities.
This module provides support for AWS S3 interconnection, as a loader, a storage and/or a result storage.
- URL: https://github.com/thumbor-community/aws
- Installing:
pip install tc_aws
To get exhaustive details about configuration options & setting it up, go to the documentation of the plugin.
thumbor_hbase (by Damien Hardy)¶
Thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.
Hbase is a column oriented database from the hadoop ecosystem.
This module provide support for Hadoop Hbase as large auto replicant key/value backend storage for images in Thumbor.
- URL: https://github.com/dhardy92/thumbor_hbase
- Installing:
pip install thumbor_hbase
Using it is simple, just change your configuration in thumbor.conf:
HBASE_STORAGE_SERVER_HOST = "localhost"
HBASE_STORAGE_SERVER_PORT = 9000
HBASE_STORAGE_TABLE = "storage-table"
HBASE_STORAGE_FAMILY = "storage-family"
If you want to use thumbor_hbase for loading original images, change your thumbor.conf to read:
LOADER = "thumbor_hbase.loader"
If you want to use thumbor_hbase for storage of original images, change your thumbor.conf to read:
STORAGE = "thumbor_hbase.storage"
thumbor_mongodb (by Damien Hardy)¶
Thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.
MongoDB is a document oriented NoSQL database.
This plugin for Thumbor is a loader that can reach images from a mongodb collection based on its Object(_id).
- URL: https://github.com/dhardy92/thumbor_mongodb
- Installing:
pip install thumbor_mongodb
Using it is simple, just change your configuration in thumbor.conf:
LOADER = 'thumbor_mongodb.loader'
MONGO_LOADER_CNX_STRING = 'mongodb://mongodbserver01,mongodbserver02:27017'
MONGO_LOADER_SERVER_DB = 'thumbor'
MONGO_LOADER_SERVER_COLLECTION = 'images'
MONGO_LOADER_DOC_FIELD = 'content'
thumbor_riak (by Damien Hardy)¶
Riak is a distributed document oriented database implementing the consistent hashing algorithm from the Dynanmo publication by Amazon.
This module provide support for Riak as a large auto replicant key/value backend storage for images in Thumbor.
- URL: https://github.com/dhardy92/thumbor_riak
- Installing:
pip install thumbor_riak(require thumbor)
Using it is simple, just change your configuration in thumbor.conf:
# Use riak for storage.
STORAGE = 'thumbor_riak.storage'
# Put the url for your riak install here
RIAK_STORAGE_BASEURL = "http://my-riak-install-base-url"
thumbor_rackspace (by David Mann)¶
This plugin allows users to store objects in the Rackspace cloud for result storage.
- URL: https://github.com/CodingNinja/thumbor_rackspace
- Installing:
pip install thumbor_rackspace
Using it is simple, just change your configuration in thumbor.conf:
# Use rackspace for result storage.
# For more info on result storage: https://github.com/thumbor/thumbor/wiki/Result-storage
RESULT_STORAGE = 'thumbor_rackspace.result_storages.cloudfile_storage'
# Pyrax Rackspace configuration file location
RACKSPACE_PYRAX_CFG = /var/thumbor/.pyrax.cfg
# Result Storage options
RACKSPACE_RESULT_STORAGE_EXPIRES = True # Set TTL on cloudfile objects
RACKSPACE_RESULT_STORAGES_CONTAINER = "cloudfile-container-name"
RACKSPACE_RESULT_STORAGES_CONTAINER_ROOT = "/"
thumbor_ceph (by Laurent Barbe)¶
Ceph a distributed object store designed to provide excellent performance, reliability and scalability.
This module provide support for Ceph RADOS as backend storage for images.
- URL: https://github.com/ksperis/thumbor_ceph
- Installing:
apt-get install python-ceph && pip install thumbor_ceph
Configuration in thumbor.conf:
################################# File Storage #################################
STORAGE = 'thumbor_ceph.storages.ceph_storage'
CEPH_STORAGE_POOL = 'thumbor'
#################################### Upload ####################################
UPLOAD_PHOTO_STORAGE = 'thumbor_ceph.storages.ceph_storage'
################################ Result Storage ################################
RESULT_STORAGE = 'thumbor_ceph.result_storages.ceph_storage'
CEPH_RESULT_STORAGE_POOL = 'thumbor'
For monitors and keys, the values used are those defined in the configuration file ceph.conf.
thumbor_spaces (by Siddhartha Mukherjee)¶
This plugin allows users to store objects in the DigitalOcean Spaces for result storage.
- URL: https://github.com/siddhartham/thumbor_spaces
- Installing:
pip install thumbor_spaces
Using it is simple, just change your configuration in thumbor.conf:
# Use DigitalOcean Spaces for result storage.
# For more info on result storage: https://github.com/thumbor/thumbor/wiki/Result-storage
RESULT_STORAGE = 'thumbor_spaces.result_storages.spaces_storage'
SPACES_REGION='xxx'
SPACES_ENDPOINT='xxx'
SPACES_KEY='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
SPACES_SECRET='XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
SPACES_BUCKET='your-bucket-name'
Metrics¶
thumbor_prometheus (by Simon Effenberg)¶
Prometheus a monitoring and alerting toolkit.
This module provide support for Prometheus as metrics collector.
- URL: https://github.com/thumbor-community/prometheus
- Installing:
pip install tc_prometheus
Configuration in thumbor.conf:
################################# Extensibility #################################
METRICS = 'tc_prometheus.metrics.prometheus_metrics'
# optional with defaults
PROMETHEUS_SCRAPE_PORT = 8000 # Port the prometheus client should listen on
Extensions¶
thumborshortener (by Thumbor Community)¶
Thumbor is a smart imaging service. It enables on-demand crop, resizing and flipping of images.
This module provides URL shortening capabilities for Thumbor. It will create an API that can shorten a thumbor URL, and then routing capabilities to reroute the shortened URL to the correct image.
The shortened URL / real URL mapping is stored within redis.
- URL: https://github.com/thumbor-community/shortener
- Installing:
pip install tc_shortener
To get exhaustive details about configuration options & setting it up, go to the documentation of the plugin.
Engines¶
thumbor-video-engine (by The Atlantic)¶
This engine extends thumbor so that it can read, crop, and transcode audio-less video files using FFmpeg. It supports input and output of animated GIF, animated WebP, WebM (VP9) video, and MP4 (H.264 and H.265).
- URL: https://github.com/theatlantic/thumbor-video-engine
- Installing:
pip install thumbor-video-engine
Configuration in thumbor.conf:
ENGINE = 'thumbor_video_engine.engines.video'
FILTERS = [
# Enables transcoding between video formats (in addition to the image
# formats already supported by thumbor.filters.format)
'thumbor_video_engine.filters.format',
# Allows outputting a still frame from a video as an image
'thumbor_video_engine.filters.still',
]
# optional, if you are already using a custom image engine
IMAGING_ENGINE = 'opencv_engine'
For a full list of configuration options and filters, read the project’s documentation.
Libraries¶
Even though the process of generating safe image URLs is explained in the Security page, we’ll try to provide libraries in each programming language to ease this process.
Available Libraries¶
Python¶
- libthumbor - Python’s extensions to thumbor. These are used to generate safe urls among others.
- django-thumbor - A django app with templatetags for resizing images with thumbor (by ricobl).
- django-thumborstorage - A Django custom storage for Thumbor backend (by Stanislas Guerra).
Node.js¶
- ThumborJS - Javascript’s extension to thumbor. These are used to generate safe urls, encrypted urls among others (by Rafael Carício).
- ThumborUrlBuilder - Thumbor client for Node JS (by David Caramelo).
- thumbor - Thumbor client for Node JS (by PolicyMic).
Ruby¶
- ruby-thumbor - Ruby’s gem to interact with thumbor server.
- thumbor_rails - Ruby’s gem to make easier to generate urls in Rails projects.
Java¶
Pollexor - Java client for the Thumbor image service which allows you to build URIs in an expressive fashion using a fluent API.
thumbor-enterprise-edition - Java library to enable generating encrypted URLs. This library is deprecated in favor of Pollexor.
PHP¶
- `Thumbor-PHP <https://github.com/beeyev/thumbor-php`__ - PHP implementation of URL generator for Thumbor. It also supports Laravel Framework.
- Phumbor - A minimal PHP client for generating Thumbor URLs.
- Phumbor for Laravel - A Laravel package providing a facade for Phumbor.
- Phumbor for Symfony2 - A Symfony2 Bundle providing a facade for Phumbor.
Objective-C¶
- OCThumbor - Objective-C for the Thumbor image service which allows you to build URIs in an expressive fashion using a fluent API.
.NET¶
- DotNetThumbor - DotNet client for the Thumbor image service. Provides an expressive fluent API.
Delphi¶
- DelphiThumbor - Delphi class to thumbor. These are used to generate safe urls among others (by Marlon Nardi).
Implementing a library¶
If you want to provide a library to enable easy usage of thumbor in your favorite programming language, please send an e-mail to thumbor@googlegroups.com and we’ll add it here.
Below are all the scenarios we think are worth testing automatically so you can guarantee compatibility with thumbor. Please note that this is not meant to be a replacement for TDD or for any other testing methodology you might want to use. These are just helper scenarios that we thought would help any library developers.
Library Tests - Generating HMAC of the URLs¶
We sincerely advise you to have thumbor installed in your machine, so you can implement a method in your tests that has thumbor generate a signature for your URL so you can compare with your own signature. This way you can make sure your url formatting and signing are working properly.
Here’s how it was implemented in Ruby:
def sign_in_thumbor(key, str)
#bash command to call thumbor's decrypt method
command = "python3 -c 'from libthumbor.url_signers.base64_hmac_sha1 import UrlSigner; signer = UrlSigner(\"" << key << "\"); print(signer.signature(\"" << str << "\").decode(\"utf-8\"))'"
#execute it in the shell using ruby's popen mechanism
result = Array.new
IO.popen(command) { |f| result.push(f.gets) }
result.join('')
end
You should be able to implement this easily in any modern programming language. It makes for very reliable tests.
Library Tests - Scenarios¶
Remember that these are in pseudo-code (BDD-like) language, and not in any programming language specifically.
Encryption Testing¶
These scenarios assume that you separate the logic of composing the url to be signed into a different “module”, that is to be tested with the URL Testing Scenarios after these scenarios.
Given
A security key of 'my-security-key'
And an image URL of "my.server.com/some/path/to/image.jpg"
And a width of 300
And a height of 200
When
I ask my library for a signed url
Then
I get '/8ammJH8D-7tXy6kU3lTvoXlhu4o=/300x200/my.server.com/some/path/to/image.jpg' as url
Given
A security key of 'my-security-key'
And an image URL of "my.server.com/some/path/to/image.jpg"
And a width of 300
And a height of 200
When
I ask my library for an encrypted URL
Then
I get the proper url (/8ammJH8D-7tXy6kU3lTvoXlhu4o=/300x200/my.server.com/some/path/to/image.jpg)
Given
A security key of 'my-security-key'
And an image URL of "my.server.com/some/path/to/image.jpg"
And the meta flag
When
I ask my library for an encrypted URL
Then
I get the proper url (/Ps3ORJDqxlSQ8y00T29GdNAh2CY=/meta/my.server.com/some/path/to/image.jpg)
Given
A security key of 'my-security-key'
And an image URL of "my.server.com/some/path/to/image.jpg"
And the smart flag
When
I ask my library for an encrypted URL
Then
I get the proper url (/-2NHpejRK2CyPAm61FigfQgJBxw=/smart/my.server.com/some/path/to/image.jpg)
Given
A security key of 'my-security-key'
And an image URL of "my.server.com/some/path/to/image.jpg"
And the fit-in flag
When
I ask my library for an encrypted URL
Then
I get the proper url (/uvLnA6TJlF-Cc-L8z9pEtfasO3s=/fit-in/my.server.com/some/path/to/image.jpg)
Given
A security key of 'my-security-key'
And an image URL of "my.server.com/some/path/to/image.jpg"
And a 'quality(20)' filter
And a 'brightness(10)' filter
When
I ask my library for an encrypted URL
Then
I get the proper url (/ZZtPCw-BLYN1g42Kh8xTcRs0Qls=/filters:brightness(10):contrast(20)/my.server.com/some/path/to/image.jpg)
You should test the same kind of tests for horizontal and vertical flip, horizontal and vertical alignment and manual cropping.
Administration¶
Configuration¶
thumbor’s configuration file is just a regular python script that gets loaded by thumbor.
In order to get a commented configuration file, just run:
thumbor-config > ./thumbor.conf
Override config through environment variable¶
It is possible overide string configs through environment variables. This is possible because thumbor uses derpconf to abstract loading configuration and derpconf allows this.
Extensibility Section¶
LOADER¶
The loader is responsible for retrieving the source image that thumbor will work with. This configuration defines the module that thumbor will use for it. This must be a full namespace module (a.k.a. python has to be able to *import* it).
LOADER = 'thumbor.loaders.http_loader'
STORAGE¶
The storage is responsible for storing the source image bytes and related metadata (face-detection, encryption and such) so that we don’t keep loading it every time. This must be a full namespace module (a.k.a. python has to be able to *import* it).
STORAGE = 'thumbor.storages.file_storage'
MIXED_STORAGE_FILE_STORAGE¶
If you are using thumbor’s mixed storage
(thumbor.storages.mixed_storage), this is where you specify the storage
that will be used to store images. This must be a full namespace
module (a.k.a. python has to be able to *import* it).
MIXED_STORAGE_FILE_STORAGE = 'thumbor.storages.file_storage'
MIXED_STORAGE_CRYPTO_STORAGE¶
If you are using thumbor’s mixed storage
(thumbor.storages.mixed_storage), this is where you specify the storage
that will be used to store cryptography information. This must be a
full namespace module (a.k.a. python has to be able to *import* it).
MIXED_STORAGE_CRYPTO_STORAGE = 'thumbor.storages.file_storage'
MIXED_STORAGE_DETECTOR_STORAGE¶
If you are using thumbor’s mixed storage
(thumbor.storages.mixed_storage), this is where you specify the storage
that will be used to store facial and feature detection results. This
must be a full namespace module (a.k.a. python has to be able to
*import* it).
MIXED_STORAGE_DETECTOR_STORAGE = 'thumbor.storages.file_storage'
RESULT_STORAGE¶
The result storage is responsible for storing the resulting image with the specified parameters (think of it as a cache), so that we don’t keep processing it every time a request comes in. This must be a full namespace module (a.k.a. python has to be able to *import* it).
RESULT_STORAGE = 'thumbor.result_storages.file_storage'
ENGINE¶
The engine is responsible for transforming the image. This must be a full namespace module (a.k.a. python has to be able to *import* it).
Currently, thumbor ships with only the thumbor.engines.pil imaging engine. A few years ago we conducted a comparison between the engines and there was no clear winner. Given PIL was the engine we were using at the time, we decided to stick with it. Other open source engines exist and you can find more about them in the plug-in section of the docs.
ENGINE = 'thumbor.engines.pil'
URL_SIGNER¶
The url signer is responsible for validation and signing of requests to prevent url tampering, which could lead to denial of service (example: filling the result_storage by specifying a different size). This must be a full namespace module (a.k.a. python has to be able to *import* it).
URL_SIGNER = 'libthumbor.url_signers.base64_hmac_sha1'
Filters Section¶
In order to specify the filters that thumbor will use, you need a
configuration key called FILTERS. This is a regular python list with the
full names (names that python can import) of the filter modules you want
to use.
i.e.:
FILTERS = [
'thumbor.filters.brightness',
'thumbor.filters.contrast',
'thumbor.filters.rgb',
'thumbor.filters.round_corner',
'thumbor.filters.quality',
'thumbor.filters.noise',
'thumbor.filters.watermark',
]
Metadata Section¶
META_CALLBACK_NAME¶
If you want thumbor to use JSONP for image metadata instead of using JSON, just set this variable to the callback name you want.
META_CALLBACK_NAME = 'thumbor_callback' # Or None for no callback
Face and Feature Detection Section¶
DETECTORS¶
This options specifies the detectors that should run the image to check for focal points.
i.e.:
DETECTORS = [
'thumbor.detectors.face_detector',
'thumbor.detectors.feature_detector'
]
Cascade Files¶
This option specifies the cascade (XML) file paths to train openCV to find faces or other objects.
## The cascade file that opencv will use to detect faces.
FACE_DETECTOR_CASCADE_FILE = 'haarcascade_frontalface_alt.xml'
## The cascade file that opencv will use to detect glasses.
GLASSES_DETECTOR_CASCADE_FILE = 'haarcascade_eye_tree_eyeglasses.xml'
## The cascade file that opencv will use to detect profile faces.
PROFILE_DETECTOR_CASCADE_FILE = 'haarcascade_profileface.xml'
Imaging Section¶
ALLOWED_SOURCES¶
This configuration defines the source of the images that thumbor will load. This is only used in the HttpLoader (check the LOADER configuration above).
ALLOWED_SOURCES=['http://s.glbimg.com']
Another example with wildcards:
ALLOWED_SOURCES=['.+\.globo\.com', '.+\.glbimg\.com']
This is to get any images that are in *.globo.com or *.glbimg.com and it
will fail with any other domains.
ACCESS_CONTROL_ALLOW_ORIGIN_HEADER¶
This allows to send the ACCESS_CONTROL_ALLOW_ORIGIN header. For example, if you want to tell the browser to allow code from any origin to access your thumbor resources:
ACCESS_CONTROL_ALLOW_ORIGIN_HEADER = '*'
If you want restrict access to a certain resource:
ACCESS_CONTROL_ALLOW_ORIGIN_HEADER = 'https://www.example.com'
Not set by default.
MAX_WIDTH and MAX_HEIGHT¶
These define the box that the resulting image for thumbor must fit-in. This means that no image that thumbor generates will have a width larger than MAX_WIDTH or height larger than MAX_HEIGHT. It defaults to 0, which means there is not limit. If the original image is larger than MAX_WIDTH x MAX_HEIGHT, it is proportionally resized to MAX_WIDTH x MAX_HEIGHT.
MAX_WIDTH = 1200
MAX_HEIGHT = 800
MIN_WIDTH and MIN_HEIGHT¶
These define the box that the resulting image for thumbor must fit-in. This means that no image that thumbor generates will have a width smaller than MIN_WIDTH or height smaller than MIN_HEIGHT. It defaults to 1. If the original image is smaller than MIN_WIDTH x MIN_HEIGHT, it is proportionally resized to MIN_WIDTH x MIN_HEIGHT.
MIN_WIDTH = 1
MIN_HEIGHT = 1
QUALITY¶
This option defines the quality that JPEG images will be generated with. It defaults to 80.
QUALITY = 90
MAX_AGE¶
This option defines the number of seconds that images should remain in the browser’s cache. It relates directly with the Expires and Cache-Control headers.
MAX_AGE = 24 * 60 * 60 # A day of caching
MAX_AGE_TEMP_IMAGE¶
When an image has some error in its detection or it has deferred queueing, it’s convenient to set a much lower expiration time for the image cache. This way the browser will request the proper image faster.
This option defines the number of seconds that images in this scenario should remain in the browser’s cache. It relates directly with the Expires and Cache-Control headers.
MAX_AGE_TEMP_IMAGE = 60 # A minute of caching
RESPECT_ORIENTATION¶
If this option is set to True, thumbor will reorient the image according to it’s EXIF Orientation tag (if one can be found). This options defaults to False.
The operations performed in the image are as follow (considering the value of the Orientation EXIF tag):
- Nothing
- Flips the image horizontally
- Rotates the image 180 degrees
- Flips the image vertically
- Flips the image vertically and rotates 270 degrees
- Rotates the image 270 degrees
- Flips the image horizontally and rotates 270 degrees
- Rotates the image 90 degrees
RESPECT_ORIENTATION = False
ALLOW_ANIMATED_GIFS¶
This option indicates whether animated gifs should be supported.
ALLOW_ANIMATED_GIFS = True
USE_GIFSICLE_ENGINE¶
This option indicates whether gifsicle should be used for all gif images, instead of the actual imaging engine. This defaults to False.
When using gifsicle thumbor will generate proper animated gifs, as well as static gifs with the smallest possible size.
USE_GIFSICLE_ENGINE = True
WARNING: When using gifsicle engine, filters will be skipped, except for cover() filter. thumbor will not do smart cropping as well.
AUTO_WEBP¶
This option indicates whether thumbor should send WebP images automatically if the request comes with an “Accept” header that specifies that the browser supports “image/webp”.
AUTO_WEBP = True
AUTO_PNG_TO_JPG¶
This option indicates whether thumbor should transform PNG images automatically to JPEG. If the image is a PNG without transparency and the numpy dependency is installed, thumbor will transform from png to jpeg. In the most of cases the image size will decrease.
WARNING: Depending on case, this is not a good deal. This transformation maybe causes distortions or the size of image can increase. Images with texts, for example, the result image maybe will be distorced. Dark images, for example, the size of result image maybe will be bigger. You have to evaluate the majority of your use cases to take a decision about the usage of this conf.
AUTO_PNG_TO_JPG = True
Queueing - Redis Single Node¶
REDIS_QUEUE_SERVER_HOST¶
Server host for the queued redis detector.
REDIS_QUEUE_SERVER_HOST = 'localhost'
REDIS_QUEUE_SERVER_DB¶
Server database index for the queued redis detector
REDIS_QUEUE_SERVER_DB = 0
REDIS_QUEUE_SERVER_PASSWORD¶
Server password for the queued redis detector
REDIS_QUEUE_SERVER_PASSWORD = None
Queueing - Redis Sentinel¶
REDIS_QUEUE_SENTINEL_INSTANCES¶
Sentinel server instances for the queued redis detector.
REDIS_QUEUE_SENTINEL_INSTANCES = 'localhost:23679,localhost:23680'
REDIS_QUEUE_SENTINEL_PASSWORD¶
Sentinel server password for the queued redis detector.
REDIS_QUEUE_SENTINEL_PASSWORD = None
REDIS_QUEUE_SENTINEL_MASTER_INSTANCE¶
Sentinel server master instance for the queued redis detector.
REDIS_QUEUE_SENTINEL_MASTER_INSTANCE = 'masterinstance'
REDIS_QUEUE_SENTINEL_MASTER_PASSWORD¶
Sentinel server master password for the queued redis detector.
REDIS_QUEUE_SENTINEL_MASTER_PASSWORD = None
REDIS_QUEUE_SENTINEL_MASTER_DB¶
Sentinel server master database index for the queued redis detector.
REDIS_QUEUE_SENTINEL_MASTER_DB = 0
REDIS_QUEUE_SENTINEL_SOCKET_TIMEOUT¶
Sentinel server socket timeout for the queued redis detector.
REDIS_QUEUE_SENTINEL_SOCKET_TIMEOUT = 10.0
Queueing - Amazon SQS¶
This queue will be removed in an upcoming release in favor of the open source AWS plug-ins for thumbor.
Security Section¶
SECURITY_KEY¶
This option specifies the security key that thumbor uses to sign secure URLs.
1234567890123456
ALLOW_UNSAFE_URL¶
This option specifies that the /unsafe url should be available in this thumbor instance. It is boolean (True or False).
Warning
It is STRONGLY recommended that you turn off this flag in production environments as this can lead to DDoS attacks against thumbor.
ALLOW_UNSAFE_URL = False
Loader Options Section¶
FILE_LOADER_ROOT_PATH¶
In case you are using thumbor’s built-in file loader, this is the option that allows you to specify where to find the images.
FILE_LOADER_ROOT_PATH = "/home/thumbor/images"
HTTP_LOADER_DEFAULT_USER_AGENT¶
This option allows users to specify the default user-agent that thumbor will send when requesting images with the HTTP Loader. Defaults to ‘thumbor/’ (like thumbor/7.0.0).
HTTP_LOADER_DEFAULT_USER_AGENT = 'thumbor/7.0.0'
HTTP_LOADER_FORWARD_USER_AGENT¶
This option tells thumbor to forward the request user agent when requesting images using the HTTP Loader. Defaults to False.
HTTP_LOADER_FORWARD_USER_AGENT = False
Storage Options Section¶
STORAGE_EXPIRATION_SECONDS¶
This options specifies the default expiration time in seconds for the storage.
STORAGE_EXPIRATION_SECONDS = 60 # 1 minute
STORES_CRYPTO_KEY_FOR_EACH_IMAGE¶
This option specifies whether thumbor should store the key for each image (thus allowing the image to be found even if the security key changes). This is a boolean flag (True or False).
Warning
If this flag is set to False, it essentially means that whenever you change the security key, for whatever reason, you just invalidated every single image that’s been generated before.
That may be ok if you have another service fetching stored images instead of allowing thumbor to do it (as many of thumbor users do).
STORAGE_CRYPTO_KEY_FOR_EACH_IMAGE = True
File Storage Section¶
FILE_STORAGE_ROOT_PATH¶
In case you are using thumbor’s built-in file storage, this is the option that allows you to specify where to save the images.
FILE_STORAGE_ROOT_PATH = '/home/thumbor/storage'
Result Storage Section¶
RESULT_STORAGE_EXPIRATION_SECONDS¶
Expiration in seconds of generated images in the result storage.
RESULT_STORAGE_EXPIRATION_SECONDS = 0
RESULT_STORAGE_FILE_STORAGE_ROOT_PATH¶
Path where the Result storage will store generated images.
RESULT_STORAGE_FILE_STORAGE_ROOT_PATH = '/tmp/thumbor/result_storage'
RESULT_STORAGE_STORES_UNSAFE¶
Indicates whether unsafe requests should also be stored in the Result Storage.
RESULT_STORAGE_STORES_UNSAFE = False
Healthcheck¶
HEALTHCHECK_ROUTE¶
The URL path to a healthcheck. This will return a 200 and the text ‘WORKING’.
HEALTHCHECK_ROUTE = '/status'
Will put the healthcheck response on http://host:port/status
The default route is /healthcheck.
Logging¶
THUMBOR_LOG_FORMAT¶
This option specifies the format to be used by logging messages sent from thumbor.
THUMBOR_LOG_FORMAT = '%(asctime)s %(name)s:%(levelname)s %(message)s'
THUMBOR_LOG_DATE_FORMAT¶
This option specifies the date format to be used by logging messages sent from thumbor.
THUMBOR_LOG_DATE_FORMAT = '%Y-%m-%d %H:%M:%S'
Error Handling¶
USE_CUSTOM_ERROR_HANDLING¶
This configuration indicates whether thumbor should use a custom error handler.
USE_CUSTOM_ERROR_HANDLING = False
ERROR_HANDLER_MODULE¶
Error reporting module. Needs to contain a class called ErrorHandler with a handle_error(context, handler, exception) method.
ERROR_HANDLER_MODULE = 'thumbor.error_handlers.sentry'
Error Handling - Sentry¶
SENTRY_DSN_URL¶
Sentry thumbor project dsn. i.e.: http://5a63d58ae7b94f1dab3dee740b301d6a:73eea45d3e8649239a973087e8f21f98@localhost:9000/2
SENTRY_DSN_URL = ''
Upload¶
UPLOAD_PHOTO_STORAGE¶
The type of storage to store uploaded images with.
UPLOAD_PHOTO_STORAGE = 'thumbor.storages.file_storage'
UPLOAD_DELETE_ALLOWED¶
Indicates whether image deletion should be allowed.
UPLOAD_DELETE_ALLOWED = False
GC_INTERVAL¶
Set manual garbage collection interval in seconds. Defaults to None (no manual garbage collection). Try this if your thumbor is running out of memory. May cause an increase in CPU load.
GC_INTERVAL = 60
Example of Configuration File¶
################################### Logging ####################################
## Logging configuration as json
## Defaults to: None
#THUMBOR_LOG_CONFIG = None
## Log Format to be used by thumbor when writing log messages.
## Defaults to: '%(asctime)s %(name)s:%(levelname)s %(message)s'
#THUMBOR_LOG_FORMAT = '%(asctime)s %(name)s:%(levelname)s %(message)s'
## Date Format to be used by thumbor when writing log messages.
## Defaults to: '%Y-%m-%d %H:%M:%S'
#THUMBOR_LOG_DATE_FORMAT = '%Y-%m-%d %H:%M:%S'
################################################################################
################################### Imaging ####################################
## Max width in pixels for images read or generated by thumbor
## Defaults to: 0
#MAX_WIDTH = 0
## Max height in pixels for images read or generated by thumbor
## Defaults to: 0
#MAX_HEIGHT = 0
## Max pixel count for images read by thumbor
## Defaults to: 75000000.0
#MAX_PIXELS = 75000000.0
## Min width in pixels for images read or generated by thumbor
## Defaults to: 1
#MIN_WIDTH = 1
## Min width in pixels for images read or generated by thumbor
## Defaults to: 1
#MIN_HEIGHT = 1
## Allowed domains for the http loader to download. These are regular
## expressions.
## Defaults to: [
#]
#ALLOWED_SOURCES = [
#]
## Quality index used for generated JPEG images
## Defaults to: 80
#QUALITY = 80
## Exports JPEG images with the `progressive` flag set.
## Defaults to: True
#PROGRESSIVE_JPEG = True
## Specify subsampling behavior for Pillow (see `subsampling` in
## http://pillow.readthedocs.org/en/latest/handbook/image-file-
## formats.html#jpeg).Be careful to use int for 0,1,2 and string for "4:4:4"
## notation. Will ignore `quality`. Using `keep` will copy the original file's
## subsampling.
## Defaults to: None
#PILLOW_JPEG_SUBSAMPLING = None
## Specify quantization tables for Pillow (see `qtables` in
## http://pillow.readthedocs.org/en/latest/handbook/image-file-
## formats.html#jpeg). Will ignore `quality`. Using `keep` will copy the
## original file's qtables.
## Defaults to: None
#PILLOW_JPEG_QTABLES = None
## Specify resampling filter for Pillow resize method.One of LANCZOS, NEAREST,
## BILINEAR, BICUBIC, HAMMING (Pillow>=3.4.0).
## Defaults to: 'LANCZOS'
#PILLOW_RESAMPLING_FILTER = 'LANCZOS'
## Quality index used for generated WebP images. If not set (None) the same level
## of JPEG quality will be used. If 100 the `lossless` flag will be used.
## Defaults to: None
#WEBP_QUALITY = None
## Compression level for generated PNG images.
## Defaults to: 6
#PNG_COMPRESSION_LEVEL = 6
## Indicates if final image should preserve indexed mode (P or 1) of original
## image
## Defaults to: True
#PILLOW_PRESERVE_INDEXED_MODE = True
## Specifies whether WebP format should be used automatically if the request
## accepts it (via Accept header)
## Defaults to: False
#AUTO_WEBP = False
## Specifies whether a PNG image should be used automatically if the png image
## has no transparency (via alpha layer). WARNING: Depending on case, this is
## not a good deal. This transformation maybe causes distortions or the size
## of image can increase. Images with texts, for example, the result image
## maybe will be distorced. Dark images, for example, the size of result image
## maybe will be bigger. You have to evaluate the majority of your use cases
## to take a decision about the usage of this conf.
## Defaults to: False
#AUTO_PNG_TO_JPG = False
## Specify the ratio between 1in and 1px for SVG images. This is only used
## whenrasterizing SVG images having their size units in cm or inches.
## Defaults to: 150
#SVG_DPI = 150
## Max AGE sent as a header for the image served by thumbor in seconds
## Defaults to: 86400
#MAX_AGE = 86400
## Indicates the Max AGE header in seconds for temporary images (images with
## failed smart detection)
## Defaults to: 0
#MAX_AGE_TEMP_IMAGE = 0
## Indicates whether thumbor should rotate images that have an Orientation EXIF
## header
## Defaults to: False
#RESPECT_ORIENTATION = False
## Ignore errors during smart detections and return image as a temp image (not
## saved in result storage and with MAX_AGE_TEMP_IMAGE age)
## Defaults to: False
#IGNORE_SMART_ERRORS = False
## Sends If-Modified-Since & Last-Modified headers; requires support from result
## storage
## Defaults to: False
#SEND_IF_MODIFIED_LAST_MODIFIED_HEADERS = False
## Sends the Access-Control-Allow-Origin header
#ACCESS_CONTROL_ALLOW_ORIGIN_HEADER = '*'
## Preserves exif information in generated images. Increases image size in
## kbytes, use with caution.
## Defaults to: False
#PRESERVE_EXIF_INFO = False
## Indicates whether thumbor should enable the EXPERIMENTAL support for animated
## gifs.
## Defaults to: True
#ALLOW_ANIMATED_GIFS = True
## Indicates whether thumbor should use gifsicle engine. Please note that smart
## cropping and filters are not supported for gifs using gifsicle (but won't
## give an error).
## Defaults to: False
#USE_GIFSICLE_ENGINE = False
## Indicates whether thumbor should enable blacklist functionality to prevent
## processing certain images.
## Defaults to: False
#USE_BLACKLIST = False
## Size of the thread pool used for image transformations. The default value is 0
## (don't use a threadpoool. Increase this if you are seeing your IOLoop
## getting blocked (often indicated by your upstream HTTP requests timing out)
## Defaults to: 0
#ENGINE_THREADPOOL_SIZE = 0
################################################################################
################################# Extensibility #################################
## The metrics backend thumbor should use to measure internal actions. This must
## be the full name of a python module (python must be able to import it)
## Defaults to: 'thumbor.metrics.logger_metrics'
#METRICS = 'thumbor.metrics.logger_metrics'
## The loader thumbor should use to load the original image. This must be the
## full name of a python module (python must be able to import it)
## Defaults to: 'thumbor.loaders.http_loader'
#LOADER = 'thumbor.loaders.http_loader'
## The file storage thumbor should use to store original images. This must be the
## full name of a python module (python must be able to import it)
## Defaults to: 'thumbor.storages.file_storage'
#STORAGE = 'thumbor.storages.file_storage'
## The result storage thumbor should use to store generated images. This must be
## the full name of a python module (python must be able to import it)
## Defaults to: None
#RESULT_STORAGE = None
## The imaging engine thumbor should use to perform image operations. This must
## be the full name of a python module (python must be able to import it)
## Defaults to: 'thumbor.engines.pil'
#ENGINE = 'thumbor.engines.pil'
## The gif engine thumbor should use to perform image operations. This must be
## the full name of a python module (python must be able to import it)
## Defaults to: 'thumbor.engines.gif'
#GIF_ENGINE = 'thumbor.engines.gif'
## The url signer thumbor should use to verify url signatures.This must be the
## full name of a python module (python must be able to import it)
## Defaults to: 'libthumbor.url_signers.base64_hmac_sha1'
#URL_SIGNER = 'libthumbor.url_signers.base64_hmac_sha1'
################################################################################
################################### Security ###################################
## The security key thumbor uses to sign image URLs
## Defaults to: 'MY_SECURE_KEY'
#SECURITY_KEY = 'MY_SECURE_KEY'
## Indicates if the /unsafe URL should be available
## Defaults to: True
#ALLOW_UNSAFE_URL = True
################################################################################
##################################### HTTP #####################################
## Enables automatically generated etags
## Defaults to: True
#ENABLE_ETAGS = True
################################################################################
################################### Storage ####################################
## Set maximum id length for images when stored
## Defaults to: 32
#MAX_ID_LENGTH = 32
################################################################################
################################# Performance ##################################
## Set garbage collection interval in seconds
## Defaults to: None
#GC_INTERVAL = None
################################################################################
################################# Healthcheck ##################################
## Healthcheck route.
## Defaults to: '/healthcheck'
#HEALTHCHECK_ROUTE = '/healthcheck'
################################################################################
################################### Metrics ####################################
## Host to send statsd instrumentation to
## Defaults to: None
#STATSD_HOST = None
## Port to send statsd instrumentation to
## Defaults to: 8125
#STATSD_PORT = 8125
## Prefix for statsd
## Defaults to: None
#STATSD_PREFIX = None
################################################################################
################################# File Loader ##################################
## The root path where the File Loader will try to find images
## Defaults to: '/home/heynemann'
#FILE_LOADER_ROOT_PATH = '/home/heynemann'
################################################################################
################################# HTTP Loader ##################################
## The maximum number of seconds libcurl can take to connect to an image being
## loaded
## Defaults to: 5
#HTTP_LOADER_CONNECT_TIMEOUT = 5
## The maximum number of seconds libcurl can take to download an image
## Defaults to: 20
#HTTP_LOADER_REQUEST_TIMEOUT = 20
## Indicates whether libcurl should follow redirects when downloading an image
## Defaults to: True
#HTTP_LOADER_FOLLOW_REDIRECTS = True
## Indicates the number of redirects libcurl should follow when downloading an
## image
## Defaults to: 5
#HTTP_LOADER_MAX_REDIRECTS = 5
## The maximum number of simultaneous HTTP connections the loader can make before
## queuing
## Defaults to: 10
#HTTP_LOADER_MAX_CLIENTS = 10
## Indicates whether thumbor should forward the user agent of the requesting user
## Defaults to: False
#HTTP_LOADER_FORWARD_USER_AGENT = False
## Indicates whether thumbor should forward the headers of the request
## Defaults to: False
#HTTP_LOADER_FORWARD_ALL_HEADERS = False
## Indicates which headers should be forwarded among all the headers of the
## request
## Defaults to: [
#]
#HTTP_LOADER_FORWARD_HEADERS_WHITELIST = [
#]
## Default user agent for thumbor http loader requests
## Defaults to: 'Thumbor/6.7.1'
#HTTP_LOADER_DEFAULT_USER_AGENT = 'Thumbor/6.7.1'
## The proxy host needed to load images through
## Defaults to: None
#HTTP_LOADER_PROXY_HOST = None
## The proxy port for the proxy host
## Defaults to: None
#HTTP_LOADER_PROXY_PORT = None
## The proxy username for the proxy host
## Defaults to: None
#HTTP_LOADER_PROXY_USERNAME = None
## The proxy password for the proxy host
## Defaults to: None
#HTTP_LOADER_PROXY_PASSWORD = None
## The filename of CA certificates in PEM format
## Defaults to: None
#HTTP_LOADER_CA_CERTS = None
## Validate the server’s certificate for HTTPS requests
## Defaults to: None
#HTTP_LOADER_VALIDATE_CERTS = None
## The filename for client SSL key
## Defaults to: None
#HTTP_LOADER_CLIENT_KEY = None
## The filename for client SSL certificate
## Defaults to: None
#HTTP_LOADER_CLIENT_CERT = None
## If the CurlAsyncHTTPClient should be used
## Defaults to: False
#HTTP_LOADER_CURL_ASYNC_HTTP_CLIENT = False
################################################################################
################################### General ####################################
## If HTTP_LOADER_CURL_LOW_SPEED_LIMIT and HTTP_LOADER_CURL_ASYNC_HTTP_CLIENT are
## set, then this is the time in seconds as integer after a download should
## timeout if the speed is below HTTP_LOADER_CURL_LOW_SPEED_LIMIT for that
## long
## Defaults to: 0
#HTTP_LOADER_CURL_LOW_SPEED_TIME = 0
## If HTTP_LOADER_CURL_LOW_SPEED_TIME and HTTP_LOADER_CURL_ASYNC_HTTP_CLIENT are
## set, then this is the limit in bytes per second as integer which should
## timeout if the speed is below that limit for
## HTTP_LOADER_CURL_LOW_SPEED_TIME seconds
## Defaults to: 0
#HTTP_LOADER_CURL_LOW_SPEED_LIMIT = 0
## Custom app class to override ThumborServiceApp. This config value is
## overridden by the -a command-line parameter.
## Defaults to: 'thumbor.app.ThumborServiceApp'
#APP_CLASS = 'thumbor.app.ThumborServiceApp'
################################################################################
################################# File Storage #################################
## Expiration in seconds for the images in the File Storage. Defaults to one
## month
## Defaults to: 2592000
#STORAGE_EXPIRATION_SECONDS = 2592000
## Indicates whether thumbor should store the signing key for each image in the
## file storage. This allows the key to be changed and old images to still be
## properly found
## Defaults to: False
#STORES_CRYPTO_KEY_FOR_EACH_IMAGE = False
## The root path where the File Storage will try to find images
## Defaults to: '/tmp/thumbor/storage'
#FILE_STORAGE_ROOT_PATH = '/tmp/thumbor/storage'
################################################################################
#################################### Upload ####################################
## Max size in bytes for images uploaded to thumbor
## Aliases: MAX_SIZE
## Defaults to: 0
#UPLOAD_MAX_SIZE = 0
## Indicates whether thumbor should enable File uploads
## Aliases: ENABLE_ORIGINAL_PHOTO_UPLOAD
## Defaults to: False
#UPLOAD_ENABLED = False
## The type of storage to store uploaded images with
## Aliases: ORIGINAL_PHOTO_STORAGE
## Defaults to: 'thumbor.storages.file_storage'
#UPLOAD_PHOTO_STORAGE = 'thumbor.storages.file_storage'
## Indicates whether image deletion should be allowed
## Aliases: ALLOW_ORIGINAL_PHOTO_DELETION
## Defaults to: False
#UPLOAD_DELETE_ALLOWED = False
## Indicates whether image overwrite should be allowed
## Aliases: ALLOW_ORIGINAL_PHOTO_PUTTING
## Defaults to: False
#UPLOAD_PUT_ALLOWED = False
## Default filename for image uploaded
## Defaults to: 'image'
#UPLOAD_DEFAULT_FILENAME = 'image'
################################################################################
################################# Mixed Storage #################################
## Mixed Storage file storage. This must be the full name of a python module
## (python must be able to import it)
## Defaults to: 'thumbor.storages.no_storage'
#MIXED_STORAGE_FILE_STORAGE = 'thumbor.storages.no_storage'
## Mixed Storage signing key storage. This must be the full name of a python
## module (python must be able to import it)
## Defaults to: 'thumbor.storages.no_storage'
#MIXED_STORAGE_CRYPTO_STORAGE = 'thumbor.storages.no_storage'
## Mixed Storage detector information storage. This must be the full name of a
## python module (python must be able to import it)
## Defaults to: 'thumbor.storages.no_storage'
#MIXED_STORAGE_DETECTOR_STORAGE = 'thumbor.storages.no_storage'
################################################################################
##################################### Meta #####################################
## The callback function name that should be used by the META route for JSONP
## access
## Defaults to: None
#META_CALLBACK_NAME = None
################################################################################
################################### Detection ###################################
## List of detectors that thumbor should use to find faces and/or features. All
## of them must be full names of python modules (python must be able to import
## it)
## Defaults to: [
#]
#DETECTORS = [
#]
## The cascade file that opencv will use to detect faces.
## Defaults to: 'haarcascade_frontalface_alt.xml'
#FACE_DETECTOR_CASCADE_FILE = 'haarcascade_frontalface_alt.xml'
## The cascade file that opencv will use to detect glasses.
## Defaults to: 'haarcascade_eye_tree_eyeglasses.xml'
#GLASSES_DETECTOR_CASCADE_FILE = 'haarcascade_eye_tree_eyeglasses.xml'
## The cascade file that opencv will use to detect profile faces.
## Defaults to: 'haarcascade_profileface.xml'
#PROFILE_DETECTOR_CASCADE_FILE = 'haarcascade_profileface.xml'
################################################################################
################################## Optimizers ##################################
## List of optimizers that thumbor will use to optimize images
## Defaults to: [
#]
#OPTIMIZERS = [
#]
## Path for the jpegtran binary
## Defaults to: '/usr/bin/jpegtran'
#JPEGTRAN_PATH = '/usr/bin/jpegtran'
## Path for the progressive scans file to use with jpegtran optimizer. Implies
## progressive jpeg output
## Defaults to: ''
#JPEGTRAN_SCANS_FILE = ''
## Path for the ffmpeg binary used to generate gifv(h.264)
## Defaults to: '/usr/local/bin/ffmpeg'
#FFMPEG_PATH = '/usr/local/bin/ffmpeg'
################################################################################
################################### Filters ####################################
## List of filters that thumbor will allow to be used in generated images. All of
## them must be full names of python modules (python must be able to import
## it)
## Defaults to: [
# 'thumbor.filters.brightness',
# 'thumbor.filters.colorize',
# 'thumbor.filters.contrast',
# 'thumbor.filters.rgb',
# 'thumbor.filters.round_corner',
# 'thumbor.filters.quality',
# 'thumbor.filters.noise',
# 'thumbor.filters.watermark',
# 'thumbor.filters.equalize',
# 'thumbor.filters.fill',
# 'thumbor.filters.sharpen',
# 'thumbor.filters.strip_exif',
# 'thumbor.filters.strip_icc',
# 'thumbor.filters.frame',
# 'thumbor.filters.grayscale',
# 'thumbor.filters.rotate',
# 'thumbor.filters.format',
# 'thumbor.filters.max_bytes',
# 'thumbor.filters.convolution',
# 'thumbor.filters.blur',
# 'thumbor.filters.extract_focal',
# 'thumbor.filters.focal',
# 'thumbor.filters.no_upscale',
# 'thumbor.filters.saturation',
# 'thumbor.filters.max_age',
# 'thumbor.filters.curve',
# 'thumbor.filters.background_color',
# 'thumbor.filters.upscale',
# 'thumbor.filters.proportion',
# 'thumbor.filters.stretch',
#]
#FILTERS = [
# 'thumbor.filters.brightness',
# 'thumbor.filters.colorize',
# 'thumbor.filters.contrast',
# 'thumbor.filters.rgb',
# 'thumbor.filters.round_corner',
# 'thumbor.filters.quality',
# 'thumbor.filters.noise',
# 'thumbor.filters.watermark',
# 'thumbor.filters.equalize',
# 'thumbor.filters.fill',
# 'thumbor.filters.sharpen',
# 'thumbor.filters.strip_exif',
# 'thumbor.filters.strip_icc',
# 'thumbor.filters.frame',
# 'thumbor.filters.grayscale',
# 'thumbor.filters.rotate',
# 'thumbor.filters.format',
# 'thumbor.filters.max_bytes',
# 'thumbor.filters.convolution',
# 'thumbor.filters.blur',
# 'thumbor.filters.extract_focal',
# 'thumbor.filters.focal',
# 'thumbor.filters.no_upscale',
# 'thumbor.filters.saturation',
# 'thumbor.filters.max_age',
# 'thumbor.filters.curve',
# 'thumbor.filters.background_color',
# 'thumbor.filters.upscale',
# 'thumbor.filters.proportion',
# 'thumbor.filters.stretch',
#]
################################################################################
################################ Result Storage ################################
## Expiration in seconds of generated images in the result storage
## Defaults to: 0
#RESULT_STORAGE_EXPIRATION_SECONDS = 0
## Path where the Result storage will store generated images
## Defaults to: '/tmp/thumbor/result_storage'
#RESULT_STORAGE_FILE_STORAGE_ROOT_PATH = '/tmp/thumbor/result_storage'
## Indicates whether unsafe requests should also be stored in the Result Storage
## Defaults to: False
#RESULT_STORAGE_STORES_UNSAFE = False
################################################################################
############################# Queued Redis Detector #############################
## Server host for the queued redis detector
## Defaults to: 'localhost'
#REDIS_QUEUE_SERVER_HOST = 'localhost'
## Server port for the queued redis detector
## Defaults to: 6379
#REDIS_QUEUE_SERVER_PORT = 6379
## Server database index for the queued redis detector
## Defaults to: 0
#REDIS_QUEUE_SERVER_DB = 0
## Server password for the queued redis detector
## Defaults to: None
#REDIS_QUEUE_SERVER_PASSWORD = None
################################################################################
############################# Queued SQS Detector ##############################
## AWS key id
## Defaults to: None
#SQS_QUEUE_KEY_ID = None
## AWS key secret
## Defaults to: None
#SQS_QUEUE_KEY_SECRET = None
## AWS SQS region
## Defaults to: 'us-east-1'
#SQS_QUEUE_REGION = 'us-east-1'
################################################################################
#################################### Errors ####################################
## This configuration indicates whether thumbor should use a custom error
## handler.
## Defaults to: False
#USE_CUSTOM_ERROR_HANDLING = False
## Error reporting module. Needs to contain a class called ErrorHandler with a
## handle_error(context, handler, exception) method.
## Defaults to: 'thumbor.error_handlers.sentry'
#ERROR_HANDLER_MODULE = 'thumbor.error_handlers.sentry'
## File of error log as json
## Defaults to: None
#ERROR_FILE_LOGGER = None
## File of error log name is parametrized with context attribute
## Defaults to: False
#ERROR_FILE_NAME_USE_CONTEXT = False
################################################################################
############################### Errors - Sentry ################################
## Sentry thumbor project dsn. i.e.: http://5a63d58ae7b94f1dab3dee740b301d6a:73ee
## a45d3e8649239a973087e8f21f98@localhost:9000/2
## Defaults to: ''
#SENTRY_DSN_URL = ''
## Sentry environment i.e.: staging
## Defaults to: None
#SENTRY_ENVIRONMENT = None
################################################################################
#################################### Server ####################################
## The amount of time to wait before shutting down the server, i.e. stop
## accepting requests.
## Defaults to: 0
#MAX_WAIT_SECONDS_BEFORE_SERVER_SHUTDOWN = 0
## The amount of time to waut before shutting down all io, after the server has
## been stopped
## Defaults to: 0
#MAX_WAIT_SECONDS_BEFORE_IO_SHUTDOWN = 0
################################################################################
Automated Error Handling¶
For many companies it make a lot of sense to have a centralized solution for handling errors in production, like sentry or squash.
Thumbor must support this type of error handling in order to better integrate itself to it’s users environments.
Enabling Custom Error Handling¶
Enabling it is as simple as setting the configuration
USE_CUSTOM_ERROR_HANDLING to True.
After that you need to set the custom error handler you want to use with
the ERROR_HANDLER_MODULE configuration. Please note that this is the
module full name, not the class full name.
Thumbor comes pre-packaged with sentry’s custom error handler:
thumbor.error_handlers.sentry. If you decide to use it, please read
below on how to configure it.
Sentry - thumbor.error_handlers.sentry¶
If you choose to use sentry custom error handler, all you need to do is
fill the SENTRY_DSN_URL configuration with sentry’s DSN URL, which
can be found in the admin page for your sentry project, like the one in
the image below:
Hosting¶
Let’s see how would you run thumbor in different environments.
Development Environment¶
For running it locally you just need to get a proper Configuration
file. You can put it at /etc/thumbor.conf, ~/thumbor.conf (home folder)
or specify it when starting thumbor.
To verify if you have thumbor, just type:
thumbor --version
It should return the version you’ve installed. Starting thumbor is as easy as:
thumbor
For more options check the Configuration page.
Production Environment¶
Other than having the proper Configuration file for your environment, we have some recommendations on how to run thumbor in production.
Our first recommendation is to run more than one instance of it. You can specify different ports using thumbor easily. This will make sure that your service stays responsive even if one of the processes die.
We also recommend having some form of load balance that distributes the
load between the aforementioned processes. We are using NGINX to do it,
but there are more sophisticated load balance softwares around. thumbor
supports health checking under the /healthcheck URI if you need to
use it.
Other than that, you run it using the thumbor console app specifying the arguments, like this:
thumbor --port=8888 --conf="~/mythumbor.conf"
We recommend using an application such as Supervisor
(http://supervisord.org/index.html) to monitor your services. An
example of a supervisord.conf file would be:
[supervisord]
logfile = /home/thumbor/logs/supervisord.log
logfile_maxbytes = 50MB
logfile_backups=10
loglevel = info
pidfile = /home/thumbor/supervisord.pid
user = thumbor
[program:thumbor]
command=thumbor --port=800%(process_num)s --conf=/etc/thumbor800%(process_num)s.conf
process_name=thumbor800%(process_num)s
numprocs=4
user=thumbor
directory=/home/thumbor/
autostart=true
autorestart=true
startretries=3
stopsignal=TERM
stdout_logfile=/home/thumbor/logs/thumbor800%(process_num)s.stdout.log
stdout_logfile_maxbytes=1MB
stdout_logfile_backups=10
stderr_logfile=/home/thumbor/logs/thumbor800%(process_num)s.stderr.log
stderr_logfile_maxbytes=1MB
stderr_logfile_backups=10
This configuration file makes sure that supervisor starts 4 processes of thumbor on the 8000, 8001, 8002 and 8003 ports, each with a different configuration file (thumbor8000.conf, thumbor8001.conf, thumbor8002.conf, thumbor8003.conf all under /etc folder). The other settings are optional, but if you need help with supervisor’s settings it has extensive documentation online (http://supervisord.org/introduction.html).
Thumbor in the Cloud¶
Running with Docker¶
Running thumbor with docker is as easy as:
$ docker run -p 8888:80 minimalcompact/thumbor
...
$ curl http://localhost:8888/healthcheck
WORKING%
For more details check the MinimalCompact thumbor docker image.
Thumbor on OpenShift¶
Warning
This may be outdated since thumbor moved to python 3.
There’s a project showing how to deploy a working version on OpenShift https://github.com/rafaelcaricio/thumbor-openshift-example
Thumbor behind CloudFront¶
Warning
This may be outdated since thumbor moved to python 3.
The awesome people at yipit are using thumbor behind the CloudFront CDN at Amazon.
The detailed information on how to do it can be seen at this blog post.
Logging¶
thumbor uses the built-in Python logging mechanisms. In order to configure log-level check the Running thumbor server page.
Configuring log format¶
Configuring the log format is as easy as including these keys in your
thumbor.conf file:
THUMBOR_LOG_FORMAT¶
Log Format to be used by thumbor when writing log messages.
Defaults to: %(asctime)s %(name)s:%(levelname)s %(message)s
THUMBOR_LOG_DATE_FORMAT¶
Date Format to be used by thumbor when writing log messages.
Defaults to: %Y-%m-%d %H:%M:%S
Running thumbor server¶
Running thumbor server is as easy as typing “thumbor” (considering you went through the proper Installing procedures).
The Server application takes some parameters that will help you tailor the thumbor Server to your needs. If you want to find out what the thumbor Server arguments are, just type:
thumbor --help
-i or –ip¶
The address that Tornado will listen for incoming request. It defaults to 0.0.0.0 (listening on localhost and current IP).
-p or –port¶
The port that Tornado will listen for incoming request. It defaults to 8888.
-c or –conf¶
The full path for the configuration file for this server.
-k or –keyfile¶
The full path for the file containing the security key to be used for this server.
-l or –log-level¶
The log level to be used. Possible values are: debug, info, warning, error, critical or notset. More on that at http://docs.python.org/library/logging.html. It defaults to warning.
–processes¶
Number of processes to run. By default equals 1 and means no forks created. Set to 0 to detect the number of cores available on this machine. Set > 1 to start that specified number of processes.
-a or –app¶
Allows the user to specify the application class to be used. This is a very advanced usage of thumbor. This argument is specified like: “namespace1.namespace2.class_name” as in “myproj.thumbor_support.MyProjThumborApp”.
Image Metadata¶
Thumbor uses py3exiv2 to read and write image metadata.
If the Py3exif2 Python library is available, the PIL engine also stores image metadata
in engine.metadata.
Reading and writing Metadata¶
This part is copied from the py3exiv2 Tutorial
Let’s retrieve a list of all the available EXIF tags available in the image:
>>> engine.metadata.exif_keys
['Exif.Image.ImageDescription',
'Exif.Image.XResolution',
'Exif.Image.YResolution',
'Exif.Image.ResolutionUnit',
'Exif.Image.Software',
'Exif.Image.DateTime',
'Exif.Image.Artist',
'Exif.Image.Copyright',
'Exif.Image.ExifTag',
'Exif.Photo.Flash',
'Exif.Photo.PixelXDimension',
'Exif.Photo.PixelYDimension']
Each of those tags can be accessed with the [] operator on the metadata,
much like a python dictionary:
>>> tag = metadata[b'Exif.Image.DateTime']
The value of an ExifTag object can be accessed in two different ways:
with the raw_value and with the value attributes:
>>> tag.raw_value
'2004-07-13T21:23:44Z'
>>> tag.value
datetime.datetime(2004, 7, 13, 21, 23, 44)
The raw value is always a byte string, this is how the value is stored in the file. The value is lazily computed from the raw value depending on the EXIF type of the tag, and is represented as a convenient python object to allow easy manipulation.
Note that querying the value of a tag may raise an ExifValueError if the
format of the raw value is invalid according to the EXIF specification (may
happen if it was written by other software that implements the specification in
a broken manner), or if pyexiv2 doesn’t know how to convert it to a convenient
python object.
Accessing the value of a tag as a python object allows easy manipulation and formatting:
>>> tag.value.strftime('%A %d %B %Y, %H:%M:%S')
'Tuesday 13 July 2004, 21:23:44'
Now let’s modify the value of the tag and write it back to the file:
>>> import datetime
>>> tag.value = datetime.datetime.today()
>>> engine.metadata.write()
Similarly to reading the value of a tag, one can set either the
raw_value or the value (which will be automatically converted to
a correctly formatted byte string by pyexiv2).
You can also add new tags to the metadata by providing a valid key and value pair (see exiv2’s documentation for a list of valid EXIF tags):
>>> key = 'Exif.Photo.UserComment'
>>> value = 'This is a useful comment.'
>>> engine.metadata[key] = pyexiv2.ExifTag(key, value)
As a handy shortcut, you can always assign a value for a given key regardless of whether it’s already present in the metadata. If a tag was present, its value is overwritten. If the tag was not present, one is created and its value is set:
>>> engine.metadata[key] = value
The EXIF data may optionally embed a thumbnail in the JPEG or TIFF format. The thumbnail can be accessed, set from a JPEG file or buffer, saved to disk and erased:
>>> thumb = engine.metadata.exif_thumbnail
>>> thumb.set_from_file('/tmp/thumbnail.jpg')
>>> thumb.write_to_file('/tmp/copy')
>>> thumb.erase()
>>> engine.metadata.write()
Installation¶
Pyexiv2 depends on the following libraries:
- boost.python (http://www.boost.org/libs/python/doc/index.html)
- exiv2 (http://www.exiv2.org/)
On OSX you can use homebrew to install the dependencies:
brew install boost boost-python3 exiv2
pip install py3exiv2
If you are updating thumbor and already have an existing virtualenv, then you have to recreate it. If you have both a System Python and a Homebrew Python with the same version, then make sure the Virtualenv uses the Homebrew Python binary.
On Linux Pyexiv2 can be installed with apt-get:
apt-get install python-all-dev libboost-python-dev libexiv2-dev pip install py3exiv2
pyexiv2.metadata API reference¶
Security¶
thumbor’s team is very concerned about security and vulnerabilities of the service. Even though the team strives to cover most scenarios, if you find any flaws or vulnerabilities, please contact the team or create an issue.
URL Tampering¶
Consider the following URL for an image:
http://some.server.com/unsafe/300x300/smart/path/to/image.jpg.
Now let’s say that some malicious user wants to overload your service. He can easily ask for other sizes in loops or worse, like:
http://some.server.com/unsafe/300x301/smart/path/to/image.jpg
http://some.server.com/unsafe/300x302/smart/path/to/image.jpg
http://some.server.com/unsafe/300x303/smart/path/to/image.jpg
...
http://some.server.com/unsafe/300x9999/smart/path/to/image.jpg
...
http://some.server.com/unsafe/9999x9999/smart/path/to/image.jpg
And that’s not even counting varying the available options.
Other than that, the user can ask for images that do not exist, thus forcing us to perform useless http GET operations or filesystem operations.
We classified both scenarios above as URL Tampering.
Stopping Tampering¶
In order to prevent users from tampering with the URL, thumbor provides
a configuration called SECURITY_KEY. This is the key used to
generate a hash-based message authentication
code.
The process is very straightforward. The web server that has the page using thumbor’s image generates an authentication code for the options and image url, using the SECURITY_KEY.
When end-users access the page and thus load the image, thumbor generates an authentication code for the same options and image url, using the same SECURITY_KEY. If both authentication codes match, thumbor processes it.
The secure endpoint looks like this:
/<authentication code with 28 characters>/300x200/smart/path/to/image.jpg.
HMAC method¶
We intend to supply toolkits in many languages that automate the signing process, but we might need help from the community in this direction.
thumbor uses standard HMAC with SHA1 signing.
Let’s use as an example the url
http://some.server.com/unsafe/300x200/smart/path/to/image.jpg.
In order to convert that to a “safe” url, we must sign the part
300x200/smart/path/to/image.jpg:
- Generate a
signatureof that part using HMAC-SHA1 with the SECURITY_KEY. - Encode the
signatureas base64. thumbor usesurlsafe_b64encodemethod of the native python’s base64 module. This method replaces some characters in the base64 string so it becomes more url friendly. - Append the
encoded_signatureto the beginning of the URL, like:/1234567890123456789012345678/300x200/smart/path/to/image.jpg.
That last part gives you the new url:
http://thumbor-server/1234567890123456789012345678/300x200/smart/path/to/image.jpg.
Notice that the url includes the options part 300x200/smart. That’s
required for thumbor to generate an authentication code to match the one
that signs the image (1234567890123456789012345678).
The code included in this documentation is illustrational and should not be used for any purposes.
The description of the base64 method is: reference
base64.urlsafe_b64encode(s)
Encode string s using a URL-safe alphabet, which substitutes
- instead of + and _ instead of / in the standard Base64 alphabet.
The result can still contain =.
Loading Images over HTTPS¶
The default http_loader loads images by default over http. To change the default to https, use the https_loader instead. To enforce https, use the strict_https_loader. Check the Image loader page for more details.
Upload¶
Warning
The upload module will be removed from thumbor’s codebase soon and ported to an extension.
How to upload Images¶
Thumbor provides a /image REST end-point to upload your images and
manage it.
This way you can send thumbor your original images by doing a simple post to its urls.
Configuration¶
The table below show all configuration parameters to manage image upload:
| Configuration parameter | Default | Description |
|---|---|---|
| UPLOAD_ENABLED | False | Indicates whether thumbor should enable File uploads |
| UPLOAD_PUT_ALLOWED | False | Indicates whether image overwrite should be allowed |
| UPLOAD_DELETE_ALLOWED | False | Indicates whether image deletion should be allowed |
| UPLOAD_PHOTO_STORAGE | thumbor.storages.file_storage | The type of storage to store uploaded images with |
| UPLOAD_DEFAULT_FILENAME | image | Default filename for image uploaded |
| UPLOAD_MAX_SIZE | 0 | Max size in Kb for images uploaded to thumbor |
| MIN_WIDTH | 1 | Min width in pixels for images uploaded |
| MIN_HEIGHT | 1 | Min height in pixels for images uploaded |
Thumbor comes with the /image REST end-point to upload disabled by
default. In order to enable it, just set the UPLOAD_ENABLED
configuration in your thumbor.conf file to True.
Thumbor will then use the storage specified in the
UPLOAD_PHOTO_STORAGE configuration to save your images. You can use
an existing storage (filesystem, redis, mongo, hbase…) or
create your own storage if needed .
You can manage image putting and deletions simply set the configuration
parameters UPLOAD_PUT_ALLOWED and UPLOAD_DELETE_ALLOWED to
True. This parameters are set to False by default for security
reasons.
Finally the upload constraints (max size, image width and height) will
be controlled by UPLOAD_MAX_SIZE, MIN_WIDTH and MIN_HEIGHT
parameters.
API Usage¶
The Thumbor /image REST end-point supports the commons HTTP
methods :
- POST : to upload a new image
- GET : to display an image uploaded
- PUT : to replace an image uploaded by another preserving the URI
- DELETE : to remove an image uploaded from storage
By default, PUT and DELETE methods are disabled as explained
above. This is done to tighten thumbor’s security.
Posting¶
Posting is the only method allowed by default when you activate the upload module. It allows new images to be sent to Thumbor.
- In order to upload a new image, you have two choices:
- send an HTTP POST to the
/imageend-point with the image as payload (REST style) - send an HTTP POST to the
/imageend-point with a multi-part form with a file field called media (Form style).
- send an HTTP POST to the
In the REST style mode you may add an optional Slug header to define
the image filename, which is useful for SEO reasons. Not specifying a
Slug causes the server to use the default filename for the image
(UPLOAD_DEFAULT_FILENAME parameter) .
The HTTP response will return a Location header pointing on the
uploaded image. The URI presented in Location header may be used to
update or delete the image uploaded (see below).
For examples, see the Upload an image via the REST API or Upload an image via a form sections.
The status code returned will be :
- 201 Created (success)
- 415 Unsupported Media Type (image type is not allowed)
- 412 Precondition Failed (image is too small or the file is not an image)
Putting¶
Putting is a little more dangerous if you don’t have strict control of
who can access the /image end-point. This is because whatever is
sent using this method gets saved to storage, overwriting the previous
entry.
In order to replace an existing image, all you have to do is send an
HTTP PUT request to the /image end-point with the new image
content as payload. The new image will replace the original image
preserving the URI.
As for the POST method you may add an optional Slug header to
define the image filename.
The HTTP response will return a Location header pointing on the
modified image. The URI presents in Location header may be used to
update again the image or delete it.
For an example see the Modifying an image section.
The status code returned will be :
- 204 No Content (success)
- 405 Method Not Allowed (if thumbor’s configuration disallows putting images)
- 415 Unsupported Media Type (image type is not allowed)
- 412 Precondition Failed (image is too small or file is not an image)
Deleting¶
Deleting can be very dangerous, thus is disabled by default.
If you do enable it, in order to delete an image, all you have to do is
send an HTTP DELETE request to the /image end-point.
For an example, see the Deleting an image section.
- 204 No Content (success)
- 404 Not Found (image doesn’t exists)
- 405 Method Not Allowed (if thumbor’s configuration disallows deleting images)
Example¶
Assuming the thumbor server is located at : http://thumbor-server
Upload an image via the REST API¶
When using the /image REST end-point to upload your image via the
REST API :
curl -i -H "Content-Type: image/jpeg" -H "Slug: photo.jpg"
-XPOST http://thumbor-server/image --data-binary "@tests/fixtures/images/20x20.jpg"
the HTTP POST request was send to the server :
POST /image
Content-Type: image/jpeg
Content-Length: 822
Slug : photo.jpg
and the Thumbor server should return :
HTTP/1.1 201 Created
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Location: /image/05b2eda857314e559630c6f3334d818d/photo.jpg
Server: TornadoServer/2.1.1
The image is created at
http://thumbor-server/image/05b2eda857314e559630c6f3334d818d/photo.jpg.
It can be retrieved, modified or deleted via this URI.
The optional Slug HTTP header specifies the filename to use for the
image uploaded.
Upload an image via a form¶
When using the /image REST end-point to upload your images via a
form, the user is free to choose the filename of the image via the
filename field :
curl -i -XPOST http://thumbor-server/image
-F "media=@tests/fixtures/images/20x20.jpg;type=image/jpeg;filename=croco.jpg"
the HTTP POST request was send to the server :
POST /image
Content-Type: multipart/form-data; boundary=----------------------------11df125d8b12
Content-Length: 822
and the Thumbor server should return :
HTTP/1.1 201 Created
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Location: /image/05b2eda857314e559630c6f3334d818d/croco.jpg
The image is created at
http://thumbor-server/image/05b2eda857314e559630c6f3334d818d/croco.jpg.
It can be retrieve, modify or delete via this URI using the REST API.
Modifying an image¶
To replace the previously uploaded image by another we use:
curl -i -H "Content-Type: image/jpeg" -H "Slug: modified_image.jpg"
-XPUT http://thumbor-server/image/05b2eda857314e559630c6f3334d818d/photo.jpg --data-binary "@tests/fixtures/images/20x20.jpg"
the HTTP PUT request was send to the server :
PUT /image/05b2eda857314e559630c6f3334d818d/photo.jpg
Content-Type: image/jpeg
Content-Length: 822
Slug : modified_image.jpg
and the Thumbor server should return :
HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Location: /image/05b2eda857314e559630c6f3334d818d/modified_image.jpg
Server: TornadoServer/2.1.1
Deleting an image¶
Finally to delete the uploaded image we use:
curl -i -XDELETE http://thumbor-server/image/05b2eda857314e559630c6f3334d818d/modified_image.jpg
the HTTP DELETE request was send to the server :
DELETE /image/05b2eda857314e559630c6f3334d818d/modified_image.jpg
and the Thumbor server should return :
HTTP/1.1 204 No Content
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Server: TornadoServer/2.1.1
Posting, Putting and Deleting¶
thumbor original photo uploading end-point supports three different http verbs: put, post and delete.
By default, put and delete are disabled. This is done to tighten thumbor’s security. If you wish to enable them, please refer to the How to upload Images page.
Posting¶
Posting is the only allowed by default method. It allows new images to be sent to thumbor. If the same image is sent again, thumbor will raise an exception.
This is done so users can’t overwrite images with other images, possibly defacing your website.
In order to post a new image, all you have to do is send a multi-part
form with a file field called media and action of
http://{thumbor-server}/image and method of POST.
HTTP status code¶
- 201 Created (success)
- 409 Conflict (image already exists)
- 412 Precondition Failed (image is too small or the file is not an image)
Putting¶
Putting is a little more dangerous if you don’t have strict control of
who can access the /image route. This is because whatever is sent
using this method gets saved to storage, overwriting the previous entry.
In order to put a new image, all you have to do is send a multi-part
form with a file field called media and action of
http://{thumbor-server}/image and method of PUT.
HTTP status code¶
- 201 Created (success)
- 405 Method Not Allowed (if thumbor’s configuration disallows putting images)
- 412 Precondition Failed (image is too small or file is not an image)
Deleting¶
Deleting can be very dangerous, thus is disabled by default.
If you do enable it, in order to delete an image, all you have to do is
send a request to http://{thumbor-server}/image with a method of
DELETE and a field called file_path with the same path that was
used when uploading the image.
HTTP status code¶
- 200 OK (success)
- 405 Method Not Allowed (if thumbor’s configuration disallows deleting images)
Creating my own Storage¶
In order to create your own original photo storage, all you have to do
is implement a class called Storage that inherits from
thumbor.storages.BaseStorage and has three simple methods: put,
exists and remove.
put is the method that actually stores the image somewhere. It could
send the picture to a remote storage like Amazon’s S3 or it could just
save the picture to the local filesystem. This method should have a
signature of put(path, bytes) and it should return the file path
(for future reference).
exists should return if the file in the given path already exists.
This method should have a signature of exists(path) and it should
return a boolean stating if the file exists.
remove should just remove the file in the given path. This method
must be idempotent, meaning that if the file has already been removed
(or does not exist for that matter) it shouldn’t do anything on
subsequent calls. This method should have a signature of
remove(path) and does not need to return anything.
After your class has been created (and hopefully tested, lol), you just
need to modify the ORIGINAL_PHOTO_STORAGE configuration option in
your thumbor.conf file to the module where you implemented your
Storage class. Please note that thumbor must be able to import this
module, so it should be somewhere in the PYTHONPATH you started thumbor
with.
Contributors & Users¶
The team¶
These are the people that created, that work or worked in thumbor across the releases:
Founders / Committers¶
- @heynemann - Founder and active committer
- @cezarsa - Committer
- @fabiomcosta - Founder and committer
Contributors¶
Contributors can be found in the Contributors page.
Whos using it¶
http://www.globo.com - globo.com uses thumbor to generate dynamic images through all products across the portal. Around 15 billion images served per month.
http://www.properati.com.ar/ - properati is also using thumbor to generate several different sizes of their properties photos, using smart cropping to get the best possible thumbnails.
Thumbor made our lives better.
At Properati.com.ar we care a lot about user experience.
When our design team came up with a beautiful design that included thumbnails of 5 different sizes and specific cropping specs, instead of enhancing our home-made, simple thumbnail-generator process we moved to Thumbor and now, we cannot live without it.
Thanks a lot!
http://yipit.com/ - yipit now uses thumbor behind the CloudFront CDN at Amazon. Their detailed experience with setting up thumbor can be seen at this blog post.
Thumbor allows Yipit to iterate quickly on new designs without having to worry about introducing new image sizes.
On demand image generation was just too slow when integrated into our application servers, but Thumbor makes it possible.
No more going through old images and creating new thumbnails before we can roll out a new design!
Our initial Thumbor installation took less than an hour to set up, and we haven’t had to spend much time thinking about it since then.
Zach Smith - CTO
http://oony.com is using thumbor to serve thumbnail images behind Amazon’s Cloudfront CDN.
We’ve previously adapted the size of the thumbnails to what was required by our design team, forcing us to have many different versions of the images we have on our site.
With Thumbor we don’t have to worry about this anymore, and we can quickly iterate and make changes to our layouts serving the optimal image format each time.
Thumbor is awesome!
The Viadeo Group owns and operates professional social networks around the world with a total membership base of over 55 million professionals. Professionals use the networks to enhance their career prospects, discover business opportunities and build relationships with new contacts as well as to create effective online identities.
With headquarters based in Paris, the Group currently has over 450 staff with offices and teams located in 10 countries.
Viadeo is using Thumbor more and more. We used to have some home-made Java code to deliver images from http://www.viadeo.com. This code is still alive for some parts of our site.
Since the end of summer 2013, Thumbor is a reality at Viadeo. First via IOS application for member’s profile photos, then our news platform uses it for new parts of the site, taking more and more place and also some Android applications.
Thumbor helps us in migrating and decoupling applications from our storage backend. We were able to move from NFS (centralized and very sensitive to high loads) to a distributed storage like HBase, using a hbase storage plugin. Using the same technique of lazy loading (via Storage cache in thumbor) we can imagine changing our image’s storage at our convenience should apache HBase start to show deficiencies. This is really comfortable for Ops.
TypeTees is an easy-to-use iPhone app that lets you speak your mind by putting your super witty slogan into an original tee and order it immediately.
We use Thumbor to generate mobile thumbnails directly from the same large images that are sent to the t-shirt garment printer. It requires dealing with masks, feature trimming, transparent images, and replacing backgrounds to give users an easy-to-see preview of the t-shirt.
Thumbor made this possible and simple without having to write an image processor from scrap.
TypeTees was developed by www.prolificinteractive.com and you can learn more about how thumbor helped them at their engineering blog post.
At JustWatch, we’re big fans of Thumbor as well.
We’re serving it behind a CloudFront custom origin like many others, and features like WebP and smart cropping saved us huge amounts of time and bandwidth.
RideLink uses Thumbor to provide the most appropriate, and optimized, image for the platform our customer is using.
Setting it up and enhancing it was an easy task thanks to the good documentation and the available plugins.
Thanks to the team behind Thumbor for making our lifes easier!
Cheers! Erico Andrei - CTO
HeyCar is using Thumbor to optimize images for each customers device, taking network and screen sizes into account.
Our Thumbor instances run on our Kubernetes Cluster, served behind a CloudFront instance for caching purposes.
Huge Thanks to all Thumbor Contributors!
Marcelo Boeira - Software Engineer
Modalova is an online shopping website dedicated to fashion for men and women.
We use Thumbor to generate product thumbnails on our website Modalova, on the grid and also on the Product Pages.
We serve more than 2,000,000 products to our customers everyday and work with more than 10,000 brands on the fashion Market.
Our Thumbor instances are running on Heroku, behind the CDN Cloudflare.
Thanks again for this wonderful project,
Cheers!
Gabriel Kaam - CEO & Founder
How to add my site or product here¶
If you are using thumbor and your site or product is not listed here, please create an issue and we’ll include your logo and a short description on how you are using it here.
Hacking on Thumbor¶
So you want to contribute with thumbor? Welcome onboard!
There are a few things you’ll need in order to properly start hacking on it.
First step is to fork it and create your own clone of thumbor.
Dependencies¶
We seriously advise you to use virtualenv since it will keep your environment clean of thumbor’s dependencies and you can choose when to “turn them on”.
You’ll also need python >= 3.7 and python poetry.
Installing poetry should be as easy as pip install poetry, but you can find more about it in their website.
Other than that, you’ll also need redis-server <https://redis.io>` installed (for queued detector unit tests).
Initializing the Environment¶
Once you’ve created your virtualenv, and installed poetry, make sure you can use poetry:
$ poetry --version
Poetry version 1.0.3
You should see something similar. After that we just need to download all python packages with:
$ make setup
Running the Tests¶
Running the tests is as easy as:
$ make test
You should see the results of running your tests after an instant.
If you are experiencing “Too many open files” errors while running the tests, try increasing the number of open files per process, by running this command:
$ ulimit -S -n 2048
Read http://superuser.com/questions/433746/is-there-a-fix-for-the-too-many-open-files-in-system-error-on-os-x-10-7-1 for more info on this.
Linting your code¶
Please ensure that your editor is configured to use black, flake8 and pylint.
Even if that’s the case, don’t forget to run make flake pylint before commiting and fixing any issues you find. That way you won’t get a request for doing so in your PR.
Pull Requests¶
After hacking and testing your contribution, it is time to make a pull request. Make sure that your code is already integrated with the master branch of thumbor before asking for a pull request.
To add thumbor as a valid remote for your repository:
$ git remote add thumbor git://github.com/thumbor/thumbor.git
To merge thumbor’s master with your fork:
$ git pull thumbor master
If there was anything to merge, just run your tests again. If they pass, send a pull request.
Introducing a new Dependency¶
If we introduce a new dependency, the testing docker images need to be updated.
If the new dependency requires changes to the docker image, make sure to update the TestDockerfile36, TestDockerfile37, TestDockerfile38 and TestDockerfile39 files.
Then build and publish with:
make test-docker-build test-docker-publish
Remember that you must be logged in with your docker hub account and you must be part of the thumbororg <https://hub.docker.com/repository/docker/thumbororg/thumbor-test> team of administrators.
Running tests in docker¶
If you do not wish to configure your environment with thumbor’s dependencies, you can use our docker image to run tests with:
make test-docker-run
Or if you want to run a specific python version with your tests:
make test-docker-39-run
Just replace ‘39’ with the python version you want: 36, 37, 38 or 39.
Licensing¶
Thumbor is licensed under the MIT License:
The MIT License
Copyright (c) 2011 globo.com thumbor@googlegroups.com
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.