Tom Lachecki

(Tomalak Geret'kal)


How To Stop Thunderbird From Freezing When You Click On Links

Is the Thunderbird-Firefox relationship getting you down? Does clicking links occasionally result in Thunderbird totally locking up for over a minute, before Firefox bursts into life with the requested page?

Apparently this started around Thunderbird v3, and here's how to fix it all the way up to v12:

Apparently Thunderbird works best when you pretend to it that its sibling Firefox is a third-party application that needs a simple shell invocation.

Tags: , , ,
Permalink | No Comments  
Yes Alex, The Standard Doesn't Mention The Stack, And That's Fine

Alex Darby writes about "the Stack", referring to both the ISO standard and Bjarne's C++ book and complains that they both evade the concept of a stack or a heap as used for laying-out objects in memory:

But the fact that the standard text on C++ all but ignores something as core to the internal operation of C++ as the Stack is telling indeed. In my experience, this is symptomatic of the disconnect between programming language and underlying implementation that exists in the academic mindset.

In fact, it's very deliberate, and has nothing to do with academia. You shouldn't need to know about implementation details like a memory stack or heap because (a) you're writing in a programming language that abstracts those details away from you, and (b) they are implementation-specific. In Bjarne's case he's writing only about C++, not how some specific platform implements it. It's a book about the language, not the computer that compiles it into another language.

Understanding this abstraction is key to writing good, portable code. That's why he's done it and it's a good thing.

When books jump into talking about a "stack" or "heap", they are mistraining people into thinking that these are part of C++ when they are in fact programming with a combination of technologies including C++, machine code, a stick of memory, an OS… and Darby seems to have fallen into that trap, as his article gives the impression that he believes a stack is a core part of C++ itself. Indeed, Bjarne is correct to talk instead about the notion of storage duration, which is the only thing the C++ language defines on the matter.

Of course, that's not to say that such details should be forever ignored or that we should pretend that, say, an x86 machine doesn't typically use a stack. Certainly in the game development world, a programmer must be aware of lower-level implementation details. I'm not advocating some academic approach that completely removes real-world components from the equation.

It'd just be helpful if Darby didn't pretend that these lower-level details were actually part of C++ at all.

Tags: , ,
Permalink | No Comments  
Navigating Secondary Expansion in GNU Make

Struggling to get variable substitution to affect targets and dependencies in GNU Make? Here's why, and how to fix it (if you don't mind a good hack).


GNU Make is powerful and, although it's been superceded in some circles by wrappers like automake or all-out replacements like CMake and scons, for simple distributions it's still got teeth.

It's not without its pitfalls, though. In the below Makefile, not only do I have to create my own string equality function eq, but the outcome is not what I'd have expected:

# Comparison function
eq = $(and $(findstring $(1),$(2)),$(findstring $(2),$(1)))

# Variables
suf = $(if $(call eq, $(NATIVE), 1),-native,)
build_dir = build$(suf)

# Targets
.PHONY: a b

# (should yield "build")
a: $(build_dir)

# (should yield "build-native")
b: NATIVE=1
b: $(build_dir)

$(build_dir):
     @echo $@ ($(build_dir))

Outcome:

# Flag behaving properly:
~$ make a NATIVE=1
build-NATIVE build-NATIVE
~$ make b NATIVE=1
build-NATIVE build-NATIVE

# Flag not behaving properly:
~$ make a
build (build)
~$ make b
build (build-NATIVE)

In target b with no flag initially given, the sub-target build is being invoked rather than the build-NATIVE target that I'm really after, as indicated by the output of $@. Notice how $(build_dir) is correct within the rule itself.

In real life, my Makefile will accept a flag describing the target platform for the build; the default case is an ARMv5 cross-compilation, but a native build can be performed for local debugging. In addition, unit tests must be built natively as they are executed locally as part of the build process. So the idea is to override the target flag for the unit test target; then, even when mixing targets as in make check all, for the check target only a flag NATIVE is automatically set to "1" even though it was missing from the commandline invocation.

Alas, as shown above, although this works for instructions inside the encapsulated rule, the same is not true for the target itself (or its dependencies, of which we haven't written any here). This is because the Makefiles run in two phases: targets and pre-requisites are expanded in the "read-in" phase; my variable isn't reset until the "target-update" phase, in which only variables inside rules are expanded.

To work around this, we can make use of "secondary expansion". Applying it to the example above, using the details provided in the GNU Make documentation:

# Comparison function
eq = $(and $(findstring $(1),$(2)),$(findstring $(2),$(1)))

# Variables
suf = $(if $(call eq, $(NATIVE), 1),-native,)
build_dir = build$(suf)

# Targets
.PHONY: a b
.SECONDEXPANSION:

# (should yield "build")
a: $$(build_dir)

# (should yield "build-native")
b: NATIVE=1
b: $$(build_dir)

$(build_dir):
     @echo $@ ($(build_dir))

Outcome:

# Flag behaving properly:
~$ make a NATIVE=1
build-NATIVE build-NATIVE
~$ make b NATIVE=1
build-NATIVE build-NATIVE

# Flag not behaving properly:
~$ make a
build (build)
~$ make b
make: *** No rule to make target `build-foo', needed by `b'. Stop.

We're closer: target b is now at least attempting to invoke the correct sub-target build-foo. Unfortunately, secondary expansion only works for the prerequisite lists and not for target names, so in the case above the target build-foo doesn't actually exist.

The sensible thing to do at this point is to get that explicit value out of all targets and use patching matching instead or, better yet, employ AutoMake. But if the targets don't have obvious patterns and a build system switch doesn't whet your appetite today, and if you fancy a good hack, there is a complete workaround:

# create comparison function
eq = $(and $(findstring $(1),$(2)),$(findstring $(2),$(1)))

suf = $(if $(call eq, $(MYVAR), 1),-foo,)
build_dir = build$(suf)

.PHONY: a b
.SECONDEXPANSION:

# Should yield `build`
a: $$(build_dir)

# Should yield `build-foo`
b: $$(if ($$call eq,$$(MYVAR),1),$$(build_dir),)
        $(if $(call eq,$(MYVAR),1),,@make b MYVAR=1)

$(build_dir):
        @echo $@ ($(build_dir)) $(INHERITED)

Outcome:

[~]$ make a INHERITED=6
build (build) 6
[~]$ make a MYVAR=1 INHERITED=6
build-foo (build-foo) 6
[~]$ make b INHERITED=6
make[1]: Entering directory `~`
build-foo (build-foo) 6
make[1]: Entering directory `~`

By invoking make from scratch for the target tree under b, we can have our targets any way we want. Any other "local" variables are even automatically inherited by the sub-make (as demonstrated above by the INHERITED variable).

Tags: , ,
Permalink | 1 Comment  
It Is Possible To Downgrade Firefox After All

Stuck with a nasty memory leak somewhere in a large AJAX-driven application that only shows up in Firefox, and no native tools with which to diagnose it, I decided to give DynaTrace AJAX Edition a go.

DynaTrace recently added support for Firefox 11 but, unfortunately, Mozilla's ridiculous high-speed release process means the major version number was bumped just over a month later to 12. The automatic upgrades that force themselves on me when I launch Firefox(1) mean I was on Firefox 12 almost immediately, and all this put together means that I couldn't use DynaTrace.

Fortunately, we can downgrade Firefox by locating the desired release from Mozilla's FTP site(2) and installing it. As long as you don't go too far back in the release list, profiles should be reasonably safe from the downgrade; in this case, going from Firefox 12 to Firefox 11 was painless… and I got to enjoy the awesomeness of DynaTrace AJAX Edition.

Firefox will immediately begin downloading the latest version the next time you launch it, and will upgrade the time after that. So, at this point, it's worth going into Tools->Options and disabling automatic upgrades like I should have done originally.

It's time that Mozilla stopped this nonsense; incrementing the major version number so often is causing a tangible nightmare with add-on compatibility, and I cannot understand why the Firefox development team leads do not appear to care.


Note 1: Hey, Mozilla! When I launch my browser, it's because I want to visit a website, not because I want to upgrade the browser.
Note 2: "uk" stands for Ukraine, not the UK. You're looking for "en-gb", which I should already know. I mean, who was I kidding?

Tags: ,
Permalink | No Comments  
What's Different About Static Members Defined In-line?

While writing my previous post, a question came to mind.

The following is not valid:

struct T
{
   static const size_t BUF_SIZE;
   char buf[BUF_SIZE];
};

size_t T::BUF_SIZE = 256;

// error: array bound is not an integer constant

Clearly, for any instance of T, the array bound could be found in a different translation unit and T's size would be incalculable.

However, since the following compiles and links, I came to wonder whether it's really well-defined:

struct T
{
   static const size_t BUF_SIZE = 100*1024;
   char buf[BUF_SIZE];
};

My standard-fu fails me so far…

Tags: , ,
Permalink | [2] Comments  
Tomalak's Tuesday Tip #13: When Is A Scoped Lock Not A Scoped Lock?

A piece of supposedly well-written multi-threaded code was giving me a headache the other day:

struct GpsdWrapper
{
    GpsdWrapper(int gpsFd);
    ~GpsdWrapper();

    void poll_AS();
    void parse_AS();

private:

    static const size_t READ_BUFFER_SIZE = 100*1024;

    char   read_buffer[READ_BUFFER_SIZE];
    size_t buffer_pos;

    int gpsFd;

    boost::mutex  buffer_lock;
    boost::thread poll_thread;
    boost::thread parse_thread;
};

GpsdWrapper::GpsdWrapper(int gpsFd)
    : gpsFd(gpsFd)
    , poll_thread (boost::bind(&GpsdWrapper::poll_AS, this))
    , parse_thread(boost::bind(&GpsdWrapper::parse_AS, this))
{}

GpsdWrapper::~GpsdWrapper()
{
    poll_thread.interrupt();
    parse_thread.interrupt();

    poll_thread.join();
    parse_thread.join();
}

void GpsdWrapper::poll_AS()
{
    // {block signals}

    fd_set rfds;

    while (true) {
        boost::this_thread::interruption_point();

        timeval tv = get_timeout(); // {defined elsewhere}
        FD_ZERO(&rfds);
        FD_SET(gpsFd, &rfds);

        int result = select(gpsFd+1, &rfds, NULL, NULL, &tv);
        if (result > 0) {
            boost::mutex::scoped_lock(buffer_lock);

            if (read_buffer_pos == READ_BUFFER_SIZE) {
                syslog(LOG_WARNING,
                   "GPS read buffer reached maximum size; flushing it");
                read_buffer_pos = 0;
            }

            int r = read(gpsFd, read_buffer + read_buffer_pos,
                 READ_BUFFER_SIZE - read_buffer_pos);
            if (r > 0) {
                read_buffer_pos += r;
            }
            else if (r == -1) {
                syslog(LOG_NOTICE, "Read failed");
                // {disconnect}
                continue;
            }
            else if (r == 0) {
                syslog(LOG_NOTICE, "Remote client closed connection");
                // {disconnect}
                continue;
            }
        }
    }
}

void GpsdWrapper::parse_AS()
{
    // {block signals}

    while (true) {
        boost::this_thread::interruption_point();

        bool buffer_is_full;
        std::string buffer_copy;

        {
            boost::mutex::scoped_lock(buffer_lock);

            buffer_is_full = (read_buffer_pos == READ_BUFFER_SIZE);
            buffer_copy.assign(read_buffer, read_buffer_pos);
            read_buffer_pos = 0;
        }

        // {pass buffer_copy to a parser implementation}

        if (!buffer_is_full)
            usleep(300000);
    }
}

The idea here is that an instance of GpsdWrapper sits around in the calling scope, spawning two threads of its own to tick over eating/buffering data from a serial device and dispatching it to some parser, respectively. The parser has its own thread as we certainly don't want serial reads being delayed by any bottlenecks in the parsing process, and we can flush the shared buffer if such a bottleneck begins to cause a backlog of unparsed data, without causing problems on the source end.

Thread safety is allegedly guaranteed by constructing the scope-bound boost::mutex::scoped_lock objects, and all the buffer maths check out. But I was still seeing data dropping in the serial read at seemingly random intervals, implying a problem with the cursor read_buffer_pos.

It took a few hours of paid yet wasted office time to spot that my scoped_locks were in fact not scope-bound at all, because I forgot to give them names:

boost::mutex::scoped_lock(buffer_lock);
// ^ temporary object of type `scoped_lock`, lives only for the line

boost::mutex::scoped_lock l(buffer_lock);
// ^ named object with automatic storage and scope-bound lifetime

This silly little typo that I'd made identically in each case, went completely undetected and essentially meant that there would be no thread-safety in the entire mechanism.

Problem randomly found after a revitalising coffee break, I then set about trying to conjure up a way for the build process to catch mistakes like this in the future. But what would they be? GCC doesn't (and cannot be configured to) emit any warnings about this code because, in the general case, there's nothing wrong with a temporary object that is not assigned to anything and that thus dies straight away.

Indeed, even in the threading world you may use a lock as a waiting "gate" that can be safely released as soon as the gate has been passed. As it turns out, using block scope purely for the purpose of using block scope is more rare than common. Since we can't expect a toolchain to read our minds, it doesn't seem likely that we'll ever get an automated mechanism for identifying this possible error.

I guess the moral of the story is to remember this story when using scoped_locks.

Tags: , , ,
Permalink | [2] Comments