Steepness of a* and b* channels in Darktable

As many aspiring photogrpahers, I want to adjust color curves in Lab color space. Changing steepness of a* and b* curves is a popular method to boost color contrast in the image (see Dan Margulis books or this summary). It works best for abstract and some of the landscape photography.

There are few open source solutions which allow to do it. Gimp can decompose an image into three Lab layers, but it doesn't have an interactive preview, and internally Gimp represents color with only 8 bits per channel. There are also Delaboratory and Lab curves for Gimp, but they don't fit my workflow very well.

These days I do almost all my photo processing in Darktable, and wanted to be able to adjust color contrast there without using another program (Delaboratory) everytime. I knew that Darktable uses floating point representation for color channels and works in Lab already. Then I noticed, that basic Margulis' recipes are about “straight” symmetric curves anyway. So if there are only two degrees of freedom, then even a simple plugin with two sliders can be very useful. And so wrote my first plugin for Darktable, and named it “Color contrast”.

This plugin allows to control the effect with only two sliders. They have user-friendly labels “green vs magenta” and “blue vs yellow”. So this plugin can be used even by users who do not know anything about Lab. Technically, two values which the sliders control are the tangents of straight symmetric curves for the transforms in a* and b* channels respectively.

An original image in Darktable:

And after applying Color Contrast plugin:

Update: The plugin has been pushed to master branch of the Darktable git. See how to install it.

The plugin should appear in post-0.9 branch of Darktable after the release of 0.9; and I hope it gets accepted for the main line. In the meanwhile you may apply these two patches over master:

0001-colorcontrast-change-steepness-of-a-and-b-curves.patch

0002-colorcontrast-clip-a-and-b-channels-reload_defaults.patch

Or you may get the patches from the (rebasing) branch custom of my git repository: darktable-custom.

Jumping to correct tags in Emacs (and in Vim)

I use Emacs to edit Python code, and navigate the code using ETags and find-tag (M-.) function. However, I noticed, that if some modules use imports like from x import foo, then sometimes (find-tag "tagname") jumps to some random file which imports a tagname, rather than to a file where tagname is defined.

I was very frustrated. Initially I thought it was a bug in etags. It appears that it is a feature. Fortunately, there is a solution which allows to find the symbol definition (scroll down to the end of the post if impatient).

Example

This is an example which reproduces the problem. Let's suppose I have three files. A.py:

from x import foo
from y import bar
    
if __name__ == "__main__":
    foo()
    bar()

x.py:

def foo():
    print "foo"

and y.py:

def bar():
    print "bar"

Then if I generate a TAGS file as

$ etags *.py

this is what it will look like:

^L
A.py,53
from x import foo^?foo^A1,0
from y import bar^?bar^A2,18
^L
x.py,19
def foo():^?foo^A1,0
^L
y.py,19
def bar():^?bar^A1,0

Now (find-tag "foo") (M-. upon foo) jumps to A.py, rather than to x.py. The problem manifests when the filename of the module using a symbol precedes alphabetically the filename of the module where the symbol is defined, but in the first place the problem is that import lines are indexed as symbol definitions.

Solution

Initially I thought it is a bug in etags, but now I think that it is a designed behaviour, though not very user-friendly.

If we see help for find-tag function, we may notice, that it has an optional next-p parameter:

(find-tag tagname &optional next-p regexp-p)

Find tag (in current tags table) whose name contains tagname.
Select the buffer containing the tag's definition, and move point there.
The default for tagname is the expression in the buffer around or before point.

If second arg next-p is t (interactively, with prefix arg), search for
another tag that matches the last tagname or regexp used.  When there are
multiple matches for a tag, more exact matches are found first.  If next-p
is the atom `-' (interactively, with prefix arg that is a negative number
or just M--), pop back to the previous tag gone to.

So, it is possible to cycle through all the positions where the tag was found by pressing M-1 M-. or Ctrl-u M-.. To cycle back, M-- M-1 M-..

I don't think it is the best solution usability-wise. While there is some value in having import positions indexed too, I'd prefer to jump always to the definition first, rather than to a random import statement. Also, the default keyboard bindings are not very ergonomic, but at least they allow to find the definition and don't require to install anything special.

P.S. Vim and ctags are affected too. To jump to the second entry of the tag, use 2 Ctrl-] rather than Ctrl-].

How to choose a Haskell array library

Choosing an array type in Haskell is a difficult task. For one-dimensional random access data structure the vector library seems to be the optimal choice most of the time. Things are more complicated if you happen to need two- or multi-dimensional arrays (matrices), access their blocks and slices as first-class structures (like in Python), enjoy destructive updates, use some linear algebra, interoperate with C and run your code in parallel...

I've reviewed what array libraries are available on Hackage, and compiled this feature matrix.

It is not complete and not finished, please let me know if I am mistaken about some of the libraries.

Data.Array and its variants from the array library seem to be the standard choice of multi-dimensional arrays for a Haskeller. They are not good anymore when you need to write array-to-array operations, access their blocks and slices, or need some linear algebra in general. Choosing the right variant is another question.

Data.Vector from the vector library is fast and has very nice API. It is going to become part of the Haskell Platform. Unfortunately, it is not usable for multi-dimensional arrays, and it doesn't support slices (strides). Only boxed variant is parallelizable.

Data.Packed.Vector and Data.Packed.Matrix from the hmatrix library provide a very nice API, and can do almost anything, if all you need is at most two-dimensional array (a matrix). A big warning sign: hmatrix is GPL. Not a sensible LGPL, but the poisonous GPL library. Also, Data.Packed.Vector and Data.Packed.Matrix are not parallelizable as far as I can see.

Data.Vector.*, Data.Matrix.* and Data.Tensor.* from the blas library. They can do all the standard linear algebra which the BLAS level 3 can offer. Their API is designed with BLAS API in mind. I didn't check if they are interoperable with C arrays, or if the elements are unboxed, or if the operations are parallelizable. The last release was in January 2009, it is not buildable on new GHC 7 yet.

Finally, there is new Data.Array.Repa arrays from the repa library. Nice thing about them: they are designed for parallelization. Not so nice thing about them: they don't give performance advantages on GHC 6, and are not yet buildable on GHC 7. Some more important limitations of the repa: no access to strides or array blocks, no built-in linear algebra, no interoperability with C.

I didn't include in the table Vec and vect libraries, which provide special case solutions in the low-dimension arrays and low-rank matrices. They are mostly tailored towards computer graphics.

Pro Git book in ePub

I've followed Pandoc instructions, and built a e-book version of Pro Git book by Scott Chacon. If you do not want to repeat these steps yourself, you can download the result:

Scott Chacon. Pro Git. [EPUB]

Scott Chacon. Pro Git. [EPUB] (alternative link)

P.S. The author kindly asks to purchase a paper version of the book to encourage more authors and publishers to use Creative Commons licenses.

Unplot.py: from plots to tabular data

Recently, when doing backups, I noticed a script on my hard drive, which I think may be useful to someone else. It takes an image with a line plot and generates a data file for that plot. So it does an operation inverse to plotting, i.e. unplotting.

Download: unplot.py.

The script is available on bitbucket. Feel free to improve.

To run the script, you first need to decide which part of the plot image you want to scan, and what values the pixels correspond to. I prefer to use Gimp or Geeqie to find pixel coordinates.

If there are many lines on the same plot, You may also decide to colourize the line you are interested in with some distinct colour. Use Gimp if necessary. Write down the colour's HTML code.

This plot is a good point to start:

Then you can run the script against it. In the directory with the script, run:

./unplot.py "#00ff00" 0 151 0 475 5.0 824 0.09 85 /path/to/plot.png > /path/to/data.txt

The first parameter is the HTML colour code of the line to select. Then there is the values and the pixel coordinate of the bottom left corner of the plot: X value, X pixel coordinate, Y value, Y pixel coordinate. Then the same for the top right angle. And finally the name of the file with the plot. The output is redirected to the text file. Please note that the origin of digital images is usually the top left corner.

After running the script try plotting the data once again to make sure you selected the right colour and region. I went too far to the right in this example and the green letters were mistaken for the part of the plot.

In Russian: unplot.py: извлекаем табличные данные из графиков.

How to select a region by intensity range in Gimp?

I wrote this as an answer to Photo.StackExchange question, how to select by histogram range in Gimp.

Step-by-step

1. Make a copy of the layer (Layer → Duplicate Layer)
2. Select the duplicate layer, apply threshold (Colors → Threshold) to select the range of intensities.
3. In Layer → Mask → Add Layer Mask (or right click in the list of layers). Select “Grayscale copy of layer” and “Invert mask”.
4. idem: Mask to selection.
5. Hide or remove the layer with mask.

An example:

An original image. I want to select the circle:

Make a copy of the layer:

Apply threshold. Note that the area to be selected is black:

Add Layer Mask using the grayscale value of the image:

Now you've obtained an image with the mask. Everything except the black circle is transparent (we can see the bottom layer through it):

Convert the mask to selection. Switch to the original layer. The circle is selected.

JavaScript highlighter for Haskell code

In this and in my other Blogspot blog I use SHJS to highlight syntax in Haskell snippets. SHJS is a JavaScript highlighter which uses language definitions from GNU Source-highlight.

To use it, you need to put online:

• SHJS main script: sh_main.min.js,
• One or more language definitions files. Haskell one: sh_haskell.min.js,
• One of the stylesheets. (You can preview them on the main SHJS page)

You may wish to download more than one language definition and concatenate all JavaScript files together. It's OK

Then put this somewhere in the <head>:

<head>
...
<script type="text/javascript" src="http://example.com/path/to/sh_main.min.js"></script>
<script type="text/javascript" src="http://example.com/path/to/sh_haskell.min.js"></script>
<link type="text/css" rel="stylesheet" href="http://example.com/path/to/stylesheet.css">
...
</head>

And add a callback to documents' body element:

<body onload="sh_highlightDocument();">

Finally, wrap your Haskell snippets with <pre class="sh_haskell"></pre>. This is how it looks:

-- Fibonacci numbers
fibs :: [Int]
fibs = 0 : 1 : zipWith (+) fibs (tail fibs)

As far as I know, Haskell language definition has already found its way to GNU Source-highlight, but the corresponding file is not yet shipped with SHJS. So you can download it either from my source repository, or generate from the language definition file (haskell.lang) yourself.

Haskell is a quirky language to highlight, but this language definition handles it reasonably well, better than the current google-code-prettify, and much better than the current GeSHi highlighter.

Update, 2011-01-25: Suprisingly, this highlighter is still better than more recent highlight.js and an alternative language definition for SHJS by Nicolas Wu (zenzike).

So this is the complete list of JavaScript highlighters for Haskell which I know about:

• this highlighter (SHJS + Haskell language definition) (test screenshot)
• highlight.js (Haskell is supported since January 2010) (test screenshot)
• google-code-prettify (test screenshot)
• GeSHi (test screenshot)
• An alternative Haskell language definition for SHJS by Nicolas Wu (test screenshot)