sunnuntai 31. heinäkuuta 2016

Writing a file, with a bit more safety


When you're writing out, say, a current software state to file (be it save of a game, or status of a map program, or current audiobook locations), you may want to be sure that the old data is overwritten only if storing new data was successful. Your user might not be very happy that, say, location data is invalid and unfixable (due to failed map settings store), or that all locations of audiobooks and podcasts are forgotten (unfortunately latter isn't my software so I can't but request author to fix it, but alas, no activity for ages...).

Like the link above may tell you, this concerns mostly Sailfish OS and thus Qt toolkit, but same principles apply for other systems too. Some systems however make this very, very difficult as actual implementation is very carefully hidden from you and may or may not be robust enough to do this for you.

Previously, due to simple laziness, I stored settings somewhat like this:

    QString file = QStandardPaths::writableLocation(QStandardPaths::DataLocation) + "/config";
    QFile f(file);
    f.open(QIODevice::WriteOnly);
    QDataStream stream(&f);
    stream.setVersion(QDataStream::Qt_5_0);

    stream << data;
    // ...
    stream << lastData;

No close; since this is C++ this will be done automatically when function goes out of scope.

However, this will fail horribly if there is a problem, like running out of disk space or something else going wrong, and you may end up with zero-length file. And surprise surprise, something did go wrong today and I lost my map halfway through 30-mile biking trek, in middle of area I hadn't visited before. Just frickin' great.

What exactly happened with phone, I don't know. Some apps started behaving very weirdly. Eventually I restarted the phone which fixed most of the issue, but corrupted settings caused a bit more grief.

But this was sufficient motivation to fix the issue, hopefully once and for all.

To fix this, the change is relatively tiny:

    QString file = QStandardPaths::writableLocation(QStandardPaths::DataLocation) + "/config.tmp";
    // ...
    stream << lastData;

    bool stastusOk = false;
    if (stream.status() == QDataStream::Ok) {
        statusOk = true;
    }

    f.close();

    if (statusOk) {
        file = QStandardPaths::writableLocation(QStandardPaths::DataLocation) + "/config";
        QFile f2(file);
        if (f2.exists())
            f2.remove();

        f.rename(file);
    }

So, initially we create a temporary file where data is stored; once we know that the data was written successfully, that file is moved to replace the original file. Unfortunately Qt does not have function to "rename and delete target if it exists" so we have to add that extra part with f2 there. If rename then fails, this does all kinds of nasty things, but that, hopefully, will not happen. Of course you can check also the .tmp file on load if you are that paranoid (or just write alternatingly to two or three files, checking newest on load); I trust above to be sufficient. Until proven wrong, of course.

The documentation is not exactly clear whether stream status is available after closing the file, or if closing is needed, so I err'd on the side of caution as I didn't feel like experimenting.

Ignoring the details, the priciple is same for other systems too. First write temporary data, then copy over to replace original. This way the changes of data loss and corruption are minimized. If you want to get really paranoid a data checksum can also be thrown in for good measure.



torstai 28. heinäkuuta 2016

Green Energy: Somebody Else's Problem


Now, I know this is a pretty large wall of (possibly somewhat incoherent, due to few rounds of re-writing and editing) text, and you might - due to various reasons - find this difficult or off-putting to read. In that case, please skip to the very end and read just the last two chapters. I've tried to sum all this up there, hopefully in easier-to-read fastion.

Recently I saw an article about how environmental organizations declare that Sweden could fully be powered by renewable energy sources by 2020. Renewables in this context typically means pretty much solar and wind, with some hydroelectric and maybe some other other sources (biomass and whatnot) thrown in, but majority would be from those two mentioned first. And I consider this kind of thinking pretty damn dangerous, at best.

I'm certainly rooting for renewables but when something like this is said, I want to see a plan. A real plan, with details on how the required amount of energy is generated, day by day. But of course they can't provide such a plan - when ambitious claim like this is made, people behind it look at two numbers: kilowatt-hours used in a country over a year, and number of kilowatt-hours generated. And why not - by ignoring the fact that energy storage is at the moment unsolved problem these claims can be made without being completely dishonest.

See, the problem with wind and solar is that they do not provide constant about of energy. The consumption of energy, however, is relatively constant and predictable. I'm more familiar with Finland's numbers so I'll use those here instead, but Sweden's numbers should be fairly similar.

Take this figure below (I think I've used this same figure in previous post). This is from the first week of this year which was quite a cold one. And dark one, mind you - solar production is essentially zero during the winter months here, due to northern location.

The red graph is total electricity consumption. You can see that energy consumption spikes during the days and is at lowest during nights, difference here being almost 2000MW. This same day/night variance is there, every day of the year, only in summers the difference is smaller, about 1000MW.

The black graph is domestic energy production. You may notice how it is way below usage graph. This, too, is these days fairly constant (unfortunately, I might add), only the amount of deficit varies. At the moment there is not enough domestic production so we import a lot of energy from Norway (mostly hydro) and Russia (who knows how that is produced, nuclear and coal very likely).

So here we have clear pattern; energy usage is relatively constant, with some day/night variation. And we can see the consumption here rising because it was cold that week, getting colder by the day. Heating takes a lot of energy.


Here is then wind production graph from the same period. Can you spot the problem already?

Oh, by the way, this data is directly from FinGrid, who manages the Finland's energy grid and thus is very well aware of what is being used and produced - and where.


The problem is that it's highly variable. Ignore the fact that there's just 600MW produced at peak; if we scale the total wind power production up ten or twenty times, we get a graph with maximum number of 6000 or 12000MW, but the zero point doesn't move. Graph will still have roughly the same shape (remember, this is entire country's wind generation, not just some small area; although not all windmills are included so there is some extrapolating included by data provider)

During that time period the wind power consumption was below 200MW until 11th day (with minimum hovering at about 100MW for about two days and in the middle of that period, on single day - 8th - it reached about 300MW in between), rising to about 600MW after that (for few days at least). So during the coldest, most power-hungry period of the year we have five days with less than third of maximum generation from wind and practically nothing from solar.

I don't know about you but in my book this is a major problem here. Do think about this:
Finland's maximum power usage there was (and is) about that 15000MW. About 2100MW of that is from hydro right now, and there really isn't any practical way to build more of that. Industry (paper mills for example) produces about 1100MW, let's assume that remains about constant. How much solar and wind you would need to realisticly cover that difference?

Now, this peak is always during the darkest, coldest time of the year, so unless there is ridiculous amount of solar the winter production is approximately nothing

And you do have to expect long dry spells with wind, like you see above; only about one sixth of peak power. Yeah, the figures will not be pretty. Either we'll be overproducing at crazy amounts at
times with no way to actually use it all (short of some methods of producing, say, hydrogen for fuel - again something that isn't realistic in near term), or we'd be badly short. Not good either way.

Energy storage? Okay, show me the storage system that can handle, let's day, full three days of shortage of "just" constant 3000MW. If you can point me to that kind of system, please show me, I'll really want to be proven wrong here! Electric cars are out immediately - people won't be happy if their cars will drain their batteries to the grid on the first day, never being able to recharge before full supply returns three days later, so it needs to be something else.
But let's look at larger scale yet. After all, we're next to Sweden (which was originally mentioned here), with their own presumably fully renewable grid. And their power situation is about as sorry as ours - no solar, almost no wind (due to geographical location right next to us, weather locally is fairly similar), just the hydro. So they need to import energy too! (or you know, they, too, would have had massively overbuilt wind, with same issues with variance mentioned above)

Norway (next to Sweden) has vast amounts of hydro, but even that has limits - they only can export that much before they, too, would be short. Certainly not enough for both Sweden and Finland. And there is also limit on how much international grid connections can transfer energy.

Russia then? Both Finland and Sweden would be importing energy from there (Sweden doesn't have land border so they would be connected through Finland's grid), and some from south, Estonia or Denmark and their grid with continental Europe connections. And when you go past national boundaries you really don't know how the energy is produced - in this case it very well could be mostly coal, imported from a country that doesn't care. Nasty, dirty, CO2-spewing coal. There goes any claims anyone could have about "green" energy. But apparently that CO2 doesn't count as it isn't produced here - despite the reason of those emissions very clearly is here.

Funnily enough, I've been told that this is the exact situation in Germany now: By shutting down their nuclear plants they can claim to be green, but in reality, just over the border in Poland there's lots of coal plants that feed the Germany's need for electricity when green sources happen to have shortages. I haven't verified if this is true, but at this point, considering above points about problems with wind and solar, it is way too plausible to ignore. I don't know about you, but I'll take nuclear power any day over horribly dirty coal.

Now, I deliberately have kept this speculation in very near future because that can be predicted somewhat easily, being just few years away. If we go past that, there might be actually viable energy storage methods used which can help out a bit. But I have some serious trouble imagining energy storages capable of even mentioned 3000MW output over even two days, which is something that would be needed when relying on wind and solar as much as green energy enthusiasts' want us to. And in countries with larger population this figure would need to be massively higher.

In the end, by externalizing the power production you only make it somebody else's problem. And how clean or dirty that production is, is pretty much out of your control. So much for the (near-term) green dependency. Short of some major - no, huge - energy density breakthroughs, it ain't happening.

So now the TL;DR:

Now, make no mistake, I am absolutely positive that we have to make serious and even drastic reductions to the CO2 output in global level. The problem here is that current actions that are being taken are proverbially making us jump out of the frying pan and into the fire. By making the easy choices - that is, building solar and wind - we must satisfy the highly varying production levels somehow as the electricity usage is fairly constant. "Smart grid" can only do so much before reduction options there run out. Add the fact that people want to shut down nuclear power - which is currently our best way to reduce dependency on coal and other fossil fuels - and end result is that we collectively end up producing more CO2, despite all the gloating articles how some country went on a full day using renewables. Just single day you say? Well, let me see the figures for the remaining of the year then.

Really, the path we have currently taken is not a dependable way forward. 



sunnuntai 24. heinäkuuta 2016

How many PCB layers?


One of the first decisions you need to do when starting a board design is number of layers you want to use. Many simple jobs can be done with just single layer, and for DIY milling/etching that might be the easiest way. However, since you can get professionally manufactured 2-layer boards for next to nothing (link as an example, there are others), I don't even bother myself anymore.

For many purposes 2 layers is sufficient so you might not need to go higher than that. However, there is a point where designing PCB with two layers is just too much of a trouble. Maybe there are too many traces, maybe ground and power planes (well, traces) get too broken, maybe you just want to pack lots of components on a small board, maybe you want to to that nice chip with fine-pitch BGA package with lots of balls.

Next option is 4-layer board with vias fully through the board. The good thing is that you can do a lot with this setup, and this is still easily available, even for prototyping (itead, linked above, takes about $10 for 2-layer 5x5cm board; 4 layers is $65. More expensive certainly, but not that much really).

At some point even that gets too difficult, especially if you have components on both sides of the board, so you may want to go for blind or buried vias. They're great - you can have a via from layer 1 to 2, and another via (with other signal) from layer 3 to 4, at exactly same location! Imagine how much more you can fit on a board with these!

Unfortunately they come with a cost, in this case literally. For prototyping there aren't that many PCB manufacturers who want to make them, and cost can easily jump to $200+ range (and that's the cheap end) for just a few boards. That might be a bit too much for you, especially since you may need several iterations (each costing that $200) to work out issues of design.

The good thing is, however, that if you are looking for even moderate production runs (let's say 100+ units), the actual production cost difference is tiny. If 2-layer board costs, say, $4, the 4-layer with blind vias might just be $8 (with initial costs included). And with larger runs the difference gets even smaller.

And even better, going to 6 or 8 layers just barely increases the production cost. Initial costs are even higher, yes, but per board the increase might actually be very small. So, unless you are planning to do only a few boards, there really isn't any reason not to go for blind vias if they make your life easier.

So what to do if you eventually want a 4-layer board but don't want to waste too much money prototyping? One option is to make larger board initially: Say you want your final board to be 5x10 cm. You first design a two-layer board with size of 10x20cm. No need to even have all the traces there - if some signal is too difficult to route you just replace it with pads at each end and connect it with a wire. That's good enough for testing the initial design, as long as you pay attention to high-current and high-frequency traces between board parts.

Then, when you design works well enough, you can scratch the prototype PCB design and start over with different board structure, moving components to their final (or close to final) posititions, change board type and start routing it again. Yes, this adds lots of work, and must be done very carefully to avoid routing mistakes when essentially re-doing entire board, but if your monetary budget is tight but work time cheap-ish (as it often is with hobby-type projects), it just might be worth the effort.



perjantai 22. heinäkuuta 2016

Another broken tablet


Some time ago I got another new Android tablet - this time just about the cheapest one I could find with sensible specifications. This was essentially meant to be plaything for our kid so I didn't want to invest too much money as I expected it to be treated badly. So I ended up with Acer Iconia One. From start I could tell that the build quality wasn't exactly great - it felt flimsy and plasticy, in short pretty much what you'd expect from cheap device.

Well, it took about two weeks until it broke the first time. Unfortunately this happened at Thailand, in middle of our two-week stay. I have absolutely no idea what happened it but display suddenly started to show some very serious "ghosting" symptoms and became very dark. Even after full power-off and even factory reset it showed traces of image shown before these. Very weird. At home I took it back and it was replaced for warranty. No problems with that since.

Now it stopped charging completely. The USB port showed signs of bad contact already before this, but now it became completely dead. So I took a closer look:


Again, through microscope with phone camera so bad quality, but I'm pretty sure you can tell that this is one unhappy USB connector. No wonder it's dead.

I suspect that this would not be covered by warranty so I didn't even bother but took it apart. Insides weren't surprising - one PCB, battery, display with touch, speakers, cameras, microphone and that's just about it. PCBs have become smaller since the last time I opened one but all the major parts are there still.


Start from top left and proceeding down and right the connectors are: Display, touch, cameras (two of them, one facing front, one back), Micro-SD, buttons (four buttons, but only three of them are exposed outside enclosure), battery, headphone and the USB.

Bottom side is fully covered with tape (no components at all), to avoid shorts as this is placed directly on top of display module. There is combined GPS/WiFi antenna on the back cover, connected via the connectors you can see there above the audio jack.

Speakers and microphone were soldered on pads; those you can see at lower left and right side of board. Cheap, cheap...

Underneath leftmost can (which ended next to camera connectors) there was mystery BGA chip with marking 8539 (left to Flash chip on picture) I couldn't find any information about, and then there is a switching regulator - very likely to drive display backlight.

Underneath next can were Hynix Flash chip, two memory chips, Mediatek ARM processor (next to memories) and Mediatek MT6323 which appears to be power management IC.

Last can I didn't open, I expect radio to be there. So in all nothing too exciting, and really - this is pretty much what every phone or tablet has these days. More expensive devices use just a larger memory that costs few dollars (!!) more, and/on few dollars more expensive processor with a bit better specs. Maybe a bit better camera and more refined build quality.

I just can't but wonder how much of the "cheap" external feel of this is actually very deliberately planned and designed to encourage shopper to go for a bit more expensive model with "better" feel, with exactly same parts inside...

But back to the USB.

Surprisingly the USB connector was a bit sturdier than most I've seen in these devices - this actually has through-hole pins that hold it on the PCB better. Unfortunately I didn't have exactly the same connector handy, but I had one close enough to work. But first, taking the old one off.

Hint: If you have trouble desoldering lead-free solder apply some new leaded solder there first. This will bring the melting point of combined solder down a bit, making the process a bit easier. This makes especially the ground pins in middle of large ground planes so much easier to desolder cleanly.

I first wicked most of the solder away from the through holes with desoldering braid, then used hot air (at 350 degrees C) to melt solder enough to allow connector to be lifted out. After that I cleaned the holes and pads as well as I could:


The mounting hole pitch wasn't the same with new connector, so I straightened rear ground pins and then cut the one pointing towards audio connector shorter, as the partially desoldered pin you see there, middle one, right next to USB pin, is not ground but some other signal. I wouldn't want to short that out.

The small chip you see there next to USB pads is very likely some common mode filter to reduce interference. It's placed uncomfortably close to USB, making soldering the pins difficult, but with some flux, patience, steady hand and thin solder tip I managed. Not the prettiest job I've done but it works, and hopefully lasts a bit longer than the previous connector. I deliberately went a bit overboard with solder on ground pins; more solder should make the mechanical connection there a bit more harder the break with stress.


And now I have a working tablet again, with just about a hour of work.




sunnuntai 17. heinäkuuta 2016

"No, that doesn't work"


Back in 90s or so, apparently, my dad told me of some math (physics-related) he had worked out. He told me of the inputs, what was done and what was the output.

I have absolutely no memory of this, so I have to take his word for it. I would've been in my teens or so back then.

Apparently I stared blankly at nothing for a while (I still do that, apparently that's how my wife knows that I'm about to come up with some really fascinating idea - again, her words, as I typically I'm otherwise too occupied to notice) and then told him that it wouldn't work.

Well, (as told) I was right - as described it wouldn't work as he had left out one very crucial step.

I'm no math genius, really. Give me a formula and I can quickly make it work in code. Give me a problem (on a field I'm at least somewhat experienced and familiar with) and I can work out a solution. It might take a long time (and loads of reference materials), but I'll get there. But nothing too far out from my experience, thank you - there is a limit on what I'm willing to absorb at a time.

Some time ago I was asked for a G-force measurement. They had a very practical problem - a vehicle, fully loaded with massive amounts of cargo, travelling a path. Figure out maximum loading stress with certain parameters. A fun thought experiment, really. I worked out a solution, wrote some code, tested briefly that it worked and offered a solution.

I guess my idea was too good, then went for simpler solution. Ah well. But now my toolkit has been reinforced with new, fun tools to work with in future.




torstai 14. heinäkuuta 2016

Small PC boards


The small form factor computer-on-board designs aren't really as new as Raspberry and others would like you to think. They've existed for ages, albeit for different purposes - mainly for embedded systems. Take this CM-X255 board I got my hands on recently:


Once again the photo quality seems to suck. That's what I get by not bothering to look for proper camera stand before taking photos. Lazy me... Although it does also have shiny conformal coating on it, making photography even more difficult.


The board is dated 2009, is about 67x44 mm in size, and the connectors are the thickest part of it. Currents ones come with ARM processor, SDRAM, some Flash storage, Ethernet, serial ports and loads of GPIO; this seems to have just about same stuff build on it. Current pricing seems to start from less than $50 per module (in volumes, I suspect). And these might be difficult to get as individual, too, unless you commit to buying at least 100 or so at a time.

Considering what it offers the packing density is pretty impressive. I tried to figure out how many layers it has, but that damn coating makes it fairly difficult. I think I counted at least 8 layers at board edge but can't exactly be sure. I'd say 8 is absolute minimum for density like this.

One major thing with these is that they require another board to properly operate; the idea is that you can design relatively simple application-specific board with whatever peripherals/IO you happen to need, and this board plugs directly in to that, taking care of the heavy lifting. Those board-to-board connectors aren't easy to work with either for hobbyist, being small, fine-pitch surface mount things with lots of pins. And for anything non-trivial multi-layer board is pretty much required, considering number of high-frequency signals on those connectors.

All in all, not as hobbyist-friendly like Raspberry and other more modern offerings, but good reminder that system-on-board devices did exist before them, too. They just were a bit more difficult to get into, at least for us common folk.



maanantai 11. heinäkuuta 2016

Your subscription is about to expire!


I've been ordering National Geographic magazine for some ten years now, on and off, letting the subscription end and then ordering it again when they send a good enough offer on my way. Like now I got one year subscription (with some additional goodies thrown in for good measure) for something like 30€.

All this time one thing that annoys me most are those letters. I think that if you have ever subscribed to NG you know what I'm talking about. I don't know if other US magazines have similar habit of pestering their customers but here it absolutely isn't common.

"Your subscription is about to expire! Renew now or it might be too late!".

Just received one today in fact. My subscription will end in January - friggin' six months from now, and they are already sending those in. Six months! Are you guys serious, really?

Well, at least throwing those letters away doesn't take too much effort.



perjantai 8. heinäkuuta 2016

Internet and phone plans


Okay, shall twist the internet pricing knife a abit?

I used to love the mobile data plans they offered here. They were something like 70 cents per month, and any data cost around 1€ per MB or so (can't remember exactly). Those were perfect for most services that used very little data - think like reporting, alarm systems and so on. I don't think I ever went past 1€ a month with those plans.

Unfortunately those plans no longer exist. They were replaced with fixed cost plans (I would've opted out if it were possible); now it's about 3€ per month, with unlimited data throttled to 256kbps max. Not too bad, but still almost four (or three, depending the reference value) times what it used to be.

Okay, how something faster then? My operator, Elisa, has this pretty nice 3G module/WLAN adapter that costs me about 15€ a month. Unlimited (yes, really unlimited) data, at max 3G speeds (apparently maxing out at around 21Mbps). And it works perfectly at my cottage, in middle of friggin' nowhere, allowing me to stream TV shows and whatnot, to the tune of about 15GB a weekend (or so the module says).

They also are running series of ads at the moment that are making fun of some American cliches and mobile pricing policies. Youtube link to said ads. I have to admit, they are pretty fun.

Home internet? Well, it isn't fiber, unfortunately, but DSL. At the moment it rates about 14Mbps down and 0.7Mbps up. Enough for my needs, really, and is of course unlimited. I have no idea how much data I actually use monthly but I'm sure it's a lot, well past 100GB. Re-install a few games? Damn, that's easily 20 or 30 or even 40GB a pop these days... And it comes with TIVO-like service too, where I can store about 4TB of TV (albeit it's on their servers, but I don't really mind - TV shows aren't that important to me). Total cost: about 40€ a  month.

My personal phone then. Well, let's just say that its data is unlimited too, and their minimum invoice-able amount is 10€. So I pay my just-over 10€ bill every three to four months. And it works literally anywhere, no matter how remote and middle of nowhere in Finland I am. Not necessarily at 3G speeds, mind you, but it's been long time since I've experienced actual "no service" notification.

Before anyone can yell about how Finland is smaller than, say, US: It's not land area that you should look at, it's population density. And there Finland and US are fairly close. And yet US manages to offer so much worse service than our operators here...

Okay, I'll just stop here. But hopefully someone out there learns that networks can actually work better and starts asking the right questions...


sunnuntai 3. heinäkuuta 2016

Multithreaded programming - it's harder than you think


There are often referred rules about optimization, sometimes referred to as a joke, sometimes taken seriously:

1. Don't do it.
2. (Only for experts) Don't do it yet.

While multithreaded programming these days is fairly common, I do think that the rules above still do apply perfectly to multithreaded (or multiprocessor, multicore, multicomputer or multiwhatever) programming.

The tools available today are great. They offer great tools to make sure that everything works well - but that's if and only if they are used properly. The problem is that people often are lazy, take shortcuts in same of speed and sometimes just don't have any idea what they are doing. Then, for a while, things seem to be fine. Then your load grows and bad things start to happen. Sometimes those bad things are masked by, say, DBA component where database hides the complex stuff. But at some point some very weird things start to happen.

Multithreaded programming is hard. Very, very hard. Even with good tools it is easy to make mistakes, or optimizations that will bite you later on.

I first worked with multithreaded programs back in 2000 or so. Most of the complexity was actually hidden from us (programmers, that is) by tools we used, so everything worked mostly nicely. Until of course they didn't. Mysterious random crashed everywhere and whatnot. Back then CPU used only had single core so it wasn't "true" concurrent processing, but OS task switching still provided enough fun times to curse when things went bad again.

Slowly I learned about system I was working on, details of the complexity the tools hid from us and multitasking in general. Multiple processes running concurrently, each running several threads on their own, with complex asynchronous communications between them.

And then, enter C system calls that affect entire system state. Like, say, setlocale. Never, ever use those in multithreaded program. Many C functions, like mentioned setlocale, or errno or many others affect global (process-wide) state which is shared by all threads. Like, say, one background thread using "C" locale for parsing text-based network data, and UI thread using user's locale for displaying same data. Damn problem took me weeks to find (very, very rare race condition - fortunately most of that time spent was hands-off so I could get other things done) and some more to fix - for some reason I can't really recall (possibly bad C++ standard library implementation) the usually preferred solutions were out of my reach or didn't work properly so I had to do less than pretty workarounds.

Lesson learned. These days, whenever I have to deal with multiple threads or processes I do pay real good attention to synchronization, always erring on side of caution. Someone might say I err on side of paranoia, with expense of (execution) speed, but the fact is that properly made synchronization is expensive - and lock-free programming (which is even more difficult than your typical multithreaded programming) in most cases is also safety-free as some corner case will come back to bite you at some point. Unless you really, really know what you are doing. Vast majority people don't, me included, so I don't even try to do lock-free.

So again. Somewhat adapted rules for multi-threaded programming:
1. Don't.
2. (Experts only) Don't try to cut corners.



perjantai 1. heinäkuuta 2016

Login failed


Some time ago I had one customer to inform me that he couldn't log in to his account. We tried to reset his password - several times I think - with no success. So after some cursing I added temporary logging to login page handling to see what is going on.

It turned out that user was logging in from iPad, and for some reason the tablet always inserted single space character before his user name when entering login details. Why this happened still escapes me, and customer swore that when he entered the information there were no space anywhere, but whatever - I added two lines of code to remove any whitespace characters before and after user name and password (before other usual sanitization) and no more problem.

Yes, maybe the string trimming should have been there already, but once again sometimes unexpected things happen, causing unexpected things to happen. Always expect the unexpected, really...