Mutex Blocks

When I was taking a compilers course, I noticed the grammars for C/C++/Java etc. allow for blocks with no conditional or function signature at the top. I couldn’t think of what this would be used for except to make the grammar easier to describe. Later, while writing code with lots of mutual exclusion, my code was getting hard to read. My friend Logan suggested if most mutexes are small blocks of code, why not use braces to denote them? Here’s what it looks like:

sema_wait(&shared->mutex);
{
  memcpy(result, current_call->result,
              sizeof(RPCResult));
  ret = current_call->retStatus;
  current_call->next = shared->available;
  shared->available = current_call_index;
}
sema_post(&shared->mutex);

Emacs also nicely indents the blocks which really extends the readability.

As Dan has pointed out, even though the braces are being used as a visual indicator to the programmer, the braced sections are still semantically treated as blocks by the compiler; meaning a new scope is created. This is not necessarily a negative, however, because a lot of the time the new scope can make sense (similar to mutexing an entire function) or, at the very least, not hurt. Dealing with the new scope is as simple as dealing with the new scope of a for-loop. If you are writing threaded code, I hope you can deal with semantic blocks.
    None Found
  • http://saiyr.blogspot.com Dan Tang

    I’m totally stalking you. But really, I found your website from del.icio.us. From a personal standpoint, I would argue that using braces to denote something like that is a bad idea, because it’s semantically (in terms of the language) meaningless/incorrect, since you’re introducing a new scope for no reason (which may or may not decrease “maintainability”?).

    I’ll agree that it makes it more readable to a degree, but why don’t syntax highlighted comments work? Obviously that means you need to have a syntax highlighted editor, but really, who doesn’t use one for programming?

    Though, it kind of feels like Hosking made blocks like that for that very reason.

  • http://saiyr.blogspot.com Dan Tang

    I’m totally stalking you. But really, I found your website from del.icio.us. From a personal standpoint, I would argue that using braces to denote something like that is a bad idea, because it’s semantically (in terms of the language) meaningless/incorrect, since you’re introducing a new scope for no reason (which may or may not decrease “maintainability”?).

    I’ll agree that it makes it more readable to a degree, but why don’t syntax highlighted comments work? Obviously that means you need to have a syntax highlighted editor, but really, who doesn’t use one for programming?

    Though, it kind of feels like Hosking made blocks like that for that very reason.

  • http://openradix.org/ Luke Hoersten

    Dan, you’re absolutely right about the braces creating a new scope but, it’s not a pointless scope and it’s nothing that would confuse even the most novice programmers (think for-loop). All blocks hold the same scope characteristics and if these characteristics were in some way hidden, that may be misleading but, in fact, it’s perfectly logical to treat a mutually exclusive code section like a block. Think of it like making a function with the mutually exclusive code and then mutexing the whole function.

    “Mutex blocks” are not “semantically meaningless/incorrect” at all because blocks are (as part of the language) a visual indicator to the programmer that this section of code has a different context that the rest: a mutually exclusive context. Plus, the readability benefits are large enough to re-organize your code a little for the gains. Syntax highlighted comments look horrible compared to blocks because the block is more meaningful at a glance. The mutex comments can easily be lost in the other comments of the code because there is nothing to distinguish them, whereas blocks are indented. This alone is a good enough reason for me.

  • http://philharnish.com Phil Harnish

    I felt compelled to come in here and comment. My first impression was that it doesn’t really seem to have any functional use, but I thought of one and decided to comment. Imagine my surprise when I saw Dan beat me to the problem.

    Suppose you used that new variable scope to create references to the very variables you are trying to protect with the mutex. Then, when you try to use those variables outside of the scope you will get compiler errors. In Luke’s example, this would mean putting this as the first line in the mutex lock:
    RPCCallBuffer* current_call = // …
    (Something similar would be done for “shared”.)

    The idea being to make it virtually impossible to reach “thread safe” memory. And by virtually I mean “the protection is virtual”–first the programmer has to remember the mutex. (Aside: protecting C from itself? If that’s not in the spirit of “disciplined C programming” I don’t know what is!)

    When actually implemented there may be pitfalls to overcome and there may be some inefficiencies which the compiler can’t undo. Interesting idea all the same.

    @ Dan: Of course, adding structure to a natural language can also go too far. One of my own examples:
    http://ampind.dyndns.org/Sandbox/ComboBox-main.js
    XML? Links? Meta-data? Who (or what?) was supposed to read this anyways!? To be fair, I did envision an IDE but I digress…

  • http://openradix.org/ Luke Hoersten

    Like I said in my above comment, the coder must be aware of the fact that the braces are a block and they will be treated as such. As long as you’ve used a for-loop, I can’t imagine this would be a pitfall. My arbitrary code example really didn’t illustrate the scope issue and how it can be used with the “mutex blocks.” So in all fairness, Dan’s point was valid for the not-so-meaningful code example. Also, if you tried to use this newly declared RPCCallBuffer* outside of the mutex, the compiler would complain.

  • http://openradix.org/ Luke Hoersten

    All blocks hold the same scope characteristics and if these characteristics were in some way hidden, that may be misleading but, in fact, it’s perfectly logical to treat a mutually exclusive code section like a block. Think of it like making a function with the mutually exclusive code and then mutexing the whole function.

    “Mutex blocks” are not “semantically meaningless/incorrect” at all because blocks are (as part of the language) a visual indicator to the programmer that this section of code has a different context that the rest: a mutually exclusive context. Plus, the readability benefits are large enough to re-organize your code a little for the gains. Syntax highlighted comments look horrible compared to blocks because the block is more meaningful at a glance. The mutex comments can easily be lost in the other comments of the code because there is nothing to distinguish them, whereas blocks are indented. This alone is a good enough reason for me.

  • http://philharnish.com Phil Harnish

    I felt compelled to come in here and comment. My first impression was that it doesn’t really seem to have any functional use, but I thought of one and decided to comment. Imagine my surprise when I saw Dan beat me to the problem.

    Suppose you used that new variable scope to create references to the very variables you are trying to protect with the mutex. Then, when you try to use those variables outside of the scope you will get compiler errors. In Luke’s example, this would mean putting this as the first line in the mutex lock:
    RPCCallBuffer* current_call = // …
    (Something similar would be done for “shared”.)

    The idea being to make it virtually impossible to reach “thread safe” memory. And by virtually I mean “the protection is virtual”–first the programmer has to remember the mutex. (Aside: protecting C from itself? If that’s not in the spirit of “disciplined C programming” I don’t know what is!)

    When actually implemented there may be pitfalls to overcome and there may be some inefficiencies which the compiler can’t undo. Interesting idea all the same.

    @ Dan: Of course, adding structure to a natural language can also go too far. One of my own examples:
    http://ampind.dyndns.org/Sandbox/ComboBox-main.js
    XML? Links? Meta-data? Who (or what?) was supposed to read this anyways!? To be fair, I did envision an IDE but I digress…

  • http://openradix.org/ Luke Hoersten

    block and they will be treated as such. As long as you’ve used a for-loop, I can’t imagine this would be a pitfall. My arbitrary code example really didn’t illustrate the scope issue and how it can be used with the “mutex blocks.” So in all fairness, Dan’s point was valid for the not-so-meaningful code example. Also, if you tried to use this newly declared RPCCallBuffer* outside of the mutex, the compiler would complain.

  • http://saiyr.blogspot.com Dan Tang

    I wouldn’t really define brace-enclosed blocks as a visual indicator to the programmer. Without a whitespace-sensitive lexer, a compiler can’t tell what you want a block to be. So rather than it being an indicator for the programmer (hope I can use HTML or that’s gonna look ugly) I would argue that it’s for the compiler. If the difference in language semantic were simply visual, then I would agree with your idea.

    Are the readability gains really that significant over some well-placed comments and/or whitespace? Sure, it’s indented, but boldly colored comments stand out more. This is apparently rather subjective, since you think comments look horrible. Though, what about whitespace? I just double spaced between groups of statements that I thought necessary, including atomic blocks. From a purely readability standpoint, creating new blocks might be more outstanding to the author, but I think other programmers (maintainers) would find it to be confusing, rather than helpful.

    Phil’s example just adds to the idea of having maintainable code. The compiler may catch it, but that’s not the point; the point is, it’s annoying. Is the ability to introduce new scopes like that really there for reasons like this? Like you said, parsing statements without treating blocks as statements would be a lot more annoying.

    To Phil about ComboBox: I dunno what exactly you’re getting at, there. If you’re talking about your overcommenting, well…you’re overcommenting. In regards to your XML and links, I don’t think that’s overdoing it; I’m pretty sure Javadocs support this and know that XMLDocs (or whatever the hell they’re called) in Visual Studio support all that fancy stuff.

    To address your last comment (kind of, but not really): If you’re willing to declare a new scope for “mutex blocks,” why not stop there? You could surround all of the “steps” in a function with braces to make them stand out, saying, “This is one instruction in this recipe.” Do you really think that would make the code look more readable? This is also rather subjective, but I think it would look comparatively terrible. I’m sure you can make comments stand out over other comments for the “more important” stuff.

    Needs more preview box. :(

  • http://openradix.org/ Luke Hoersten

    My suggestion to use blocks for mutexes is simply for aesthetic purposes. The new scope is neither good nor bad. It’s simply there.

    The reason I suggest the use the “mutex blocks” is because its nice to think of mutex locking and unlocking as a block of code. Just as Phil’s link shows, some files (including the framework code we use) come jam packed with comments and trying to pick out which mutexes lock and unlock where is very hard.

    Using comments is just as acceptable if the rest of the code is commented cleanly. Still, the block makes it simple to see, especially with “embedded” mutexes, the order of the locks and what code is protected.

    It’s not that your points are invalid, Dan, it’s just that your conclusion about the blocks being “incorrect” is a bit stronger than what I’ve seen evidence for. The blocks have never been hard to understand scope-wise, or cumbersome. Even with clean commenting, the blocks have created much more readable code.

  • http://saiyr.blogspot.com Dan Tang

    Phil’s example is also a bad example of commonly seen code; I’ve never seen code in used in any real projects that comment every line on a new line. The closest I’ve seen is comments on the same line in assembly. I think it’s better to assume sparse/no comments than to assume over-commented code.

    Do embedded mutexes really help you see the order of your locks? What if there are (highly exaggerated) 10 mutexes you have to lock for the atomic code? That’s a helluva lot of indention, even with 2-space indention.

    The reason I think your use of lexical scoping is incorrect is this: Look at some of the languages with more advanced grammars, such as Python and Ruby. Python uses whitespace-sensitive lexing, and Ruby uses [legacy] braces or fancier stuff like do/end for for statements. Try randomly indenting in Python and see what happens, or try putting “{ print ‘hello’ }” or do “print ‘hello’ end” into irb. It doesn’t work!

    Blocks have never been hard to understand, because usually they’re associated with another statement. Do you think other programmers could look at your code and figure out why you declared a new scope immediately? This technique may work for you, but I don’t think it works well if you’re handing off code to someone you may not ever talk to.

    And to restate my question, do you think putting “steps” of a function in separate blocks will make the code more readable?

  • http://saiyr.blogspot.com Dan Tang

    programmer (hope I can use HTML or that’s gonna look ugly) I would argue that it’s for the compiler. If the difference in language semantic were simply visual, then I would agree with your idea.

    Are the readability gains really that significant over some well-placed comments and/or whitespace? Sure, it’s indented, but boldly colored comments stand out more. This is apparently rather subjective, since you think comments look horrible. Though, what about whitespace? I just double spaced between groups of statements that I thought necessary, including atomic blocks. From a purely readability standpoint, creating new blocks might be more outstanding to the author, but I think other programmers (maintainers) would find it to be confusing, rather than helpful.

    Phil’s example just adds to the idea of having maintainable code. The compiler may catch it, but that’s not the point; the point is, it’s annoying. Is the ability to introduce new scopes like that really there for reasons like this? Like you said, parsing statements without treating blocks as statements would be a lot more annoying.

    To Phil about ComboBox: I dunno what exactly you’re getting at, there. If you’re talking about your overcommenting, well…you’re overcommenting. In regards to your XML and links, I don’t think that’s overdoing it; I’m pretty sure Javadocs support this and know that XMLDocs (or whatever the hell they’re called) in Visual Studio support all that fancy stuff.

    To address your last comment (kind of, but not really): If you’re willing to declare a new scope for “mutex blocks,” why not stop there? You could surround all of the “steps” in a function with braces to make them stand out, saying, “This is one instruction in this recipe.” Do you really think that would make the code look more readable? This is also rather subjective, but I think it would look comparatively terrible. I’m sure you can make comments stand out over other comments for the “more important” stuff.

    Needs more preview box. :(

  • http://saiyr.blogspot.com Dan Tang

    woops, I didn’t include the “do” in “do print ‘hello’ end”

  • http://openradix.org/ Luke Hoersten

    1. This technique wouldn’t be used in a language with a more advanced grammar for exactly that reason: it has a more advanced grammar.
    2. If 10 mutexes needed to be locked for the same section of code, only one pair of braces would be needed.
    3. I would never use this technique for anything but course projects where arbitrary techniques may be invented to ease the readability of a not-so-well separated assignment.
    4. It is a bad idea to put “steps” of a function in separate blocks, just as it is bad to put comments above every line of code. Suggesting these extremes is absurd.

    I’m not going to answer any more questions like this because you should really be able to answer these on your own. Also, I really hope people will take these ideas and apply them appropriately: to situations that may benefit from and idea like this. If you don’t see how it can be beneficial then don’t use it.

  • http://philharnish.com Phil Harnish

    Warning: I’m about to take this way to seriously.

    What I was trying to get at with the comments was mixing the semantic meaning of a mutex lock with the visual meaning of commenting.

    I think the goal here is to solve two problems: One functional and one visual. Functionally, the goal of the code is to make a discrete portion of the code mutually exclusive (lock, modify, unlock). The other goal is to see at a glance that was achieved.

    If your means to that end is solely commenting and whitespace then the pressure is on the developer to visually interpret the functional meaning. This is what I called adding structure to a natural language. Who are you catering to when you do that?

    Things really get meta at this point, but here is how I see it…

    There is code with inherent meaning (cf. html). The code is interpreted by both compilers (cf. web browsers) and programmers (cf. web developers). Here is the part I feel strongly about: one should not compromise their own interpretation for the sake of another. That is, don’t make machine-like statements in an area which a human is expected to interpret and a machine is expected to ignore. If your problem is visual interpretation, braces aren’t the answer. Conversely, if the problem is ensuring safe, correct code no amount of comments will do that.

    That said, do I use the braces? I did for a while in a past project but stopped when I realized there wasn’t a problem to solve–neither I nor the compiler had any trouble interpreting the code. This rant was half for myself since semantics is something I feel pretty strongly about.

    fake edit: That code example I provided explored how I could achieve what I felt was lacking not only in documentation (e.g., links) but also the language (e.g., meta code blocks). (Aside: I tend to disagree with the notion of over commenting in general. Humans too can ignore comments.)

  • http://saiyr.blogspot.com Dan Tang

    1. My point was simply that you’re using “a hole” in the grammar, something I don’t agree with.

    2. What if they weren’t locked directly next to each other, for some reason?

    3. Well, if your blog post had said that, I probably wouldn’t have bothered saying anything. We all do shady things in CS projects ;)

    4. I don’t mean every line of code should have a block. What I mean is, one instruction to another human might correspond to more than one line of code. So does it seem better to put those lines of code into a block to clarify that it is a single step to a human?

    Anyway, you come off as getting irritated, so I’ll just stop commenting here. The misunderstanding to begin with was that I thought you would use this outside of class.

  • http://openradix.org/ Luke Hoersten

    My suggestion to use blocks for mutexes is simply for aesthetic purposes. The new scope is neither good nor bad. It’s simply there.

    The reason I suggest the use the “mutex blocks” is because its nice to think of mutex locking and unlocking as a block of code. Just as Phil’s link shows, some files (including the framework code we use) come jam packed with comments and trying to pick out which mutexes lock and unlock where is very hard.

    Using comments is just as acceptable if the rest of the code is commented cleanly. Still, the block makes it simple to see, especially with “embedded” mutexes, the order of the locks and what code is protected.

    It’s not that your points are invalid, Dan, it’s just that your conclusion about the blocks being “incorrect” is a bit stronger than what I’ve seen evidence for. The blocks have never been hard to understand scope-wise, or cumbersome. Even with clean commenting, the blocks have created much more readable code.

  • http://saiyr.blogspot.com Dan Tang

    Phil’s example is also a bad example of commonly seen code; I’ve never seen code in used in any real projects that comment every line on a new line. The closest I’ve seen is comments on the same line in assembly. I think it’s better to assume sparse/no comments than to assume over-commented code.

    Do embedded mutexes really help you see the order of your locks? What if there are (highly exaggerated) 10 mutexes you have to lock for the atomic code? That’s a helluva lot of indention, even with 2-space indention.

    The reason I think your use of lexical scoping is incorrect is this: Look at some of the languages with more advanced grammars, such as Python and Ruby. Python uses whitespace-sensitive lexing, and Ruby uses [legacy] braces or fancier stuff like do/end for for statements. Try randomly indenting in Python and see what happens, or try putting “{ print ‘hello’ }” or do “print ‘hello’ end” into irb. It doesn’t work!

    Blocks have never been hard to understand, because usually they’re associated with another statement. Do you think other programmers could look at your code and figure out why you declared a new scope immediately? This technique may work for you, but I don’t think it works well if you’re handing off code to someone you may not ever talk to.

    And to restate my question, do you think putting “steps” of a function in separate blocks will make the code more readable?

  • http://saiyr.blogspot.com Dan Tang

    woops, I didn’t include the “do” in “do print ‘hello’ end”

  • http://openradix.org/ Luke Hoersten

    1. This technique wouldn’t be used in a language with a more advanced grammar for exactly that reason: it has a more advanced grammar.
    2. If 10 mutexes needed to be locked for the same section of code, only one pair of braces would be needed.
    3. I would never use this technique for anything but course projects where arbitrary techniques may be invented to ease the readability of a not-so-well separated assignment.
    4. It is a bad idea to put “steps” of a function in separate blocks, just as it is bad to put comments above every line of code. Suggesting these extremes is absurd.

    I’m not going to answer any more questions like this because you should really be able to answer these on your own. Also, I really hope people will take these ideas and apply them appropriately: to situations that may benefit from and idea like this. If you don’t see how it can be beneficial then don’t use it.

  • http://philharnish.com Phil Harnish

    semantic meaning of a mutex lock with the visual meaning of commenting.

    I think the goal here is to solve two problems: One functional and one visual. Functionally, the goal of the code is to make a discrete portion of the code mutually exclusive (lock, modify, unlock). The other goal is to see at a glance that was achieved.

    If your means to that end is solely commenting and whitespace then the pressure is on the developer to visually interpret the functional meaning. This is what I called adding structure to a natural language. Who are you catering to when you do that?

    Things really get meta at this point, but here is how I see it…

    There is code with inherent meaning (cf. html). The code is interpreted by both compilers (cf. web browsers) and programmers (cf. web developers). Here is the part I feel strongly about: one should not compromise their own interpretation for the sake of another. That is, don’t make machine-like statements in an area which a human is expected to interpret and a machine is expected to ignore. If your problem is visual interpretation, braces aren’t the answer. Conversely, if the problem is ensuring safe, correct code no amount of comments will do that.

    That said, do I use the braces? I did for a while in a past project but stopped when I realized there wasn’t a problem to solve–neither I nor the compiler had any trouble interpreting the code. This rant was half for myself since semantics is something I feel pretty strongly about.

    fake edit: That code example I provided explored how I could achieve what I felt was lacking not only in documentation (e.g., links) but also the language (e.g., meta code blocks). (Aside: I tend to disagree with the notion of over commenting in general. Humans too can ignore comments.)

  • http://saiyr.blogspot.com Dan Tang

    1. My point was simply that you’re using “a hole” in the grammar, something I don’t agree with.

    2. What if they weren’t locked directly next to each other, for some reason?

    3. Well, if your blog post had said that, I probably wouldn’t have bothered saying anything. We all do shady things in CS projects ;)

    4. I don’t mean every line of code should have a block. What I mean is, one instruction to another human might correspond to more than one line of code. So does it seem better to put those lines of code into a block to clarify that it is a single step to a human?

    Anyway, you come off as getting irritated, so I’ll just stop commenting here. The misunderstanding to begin with was that I thought you would use this outside of class.

  • Trevor Wernisch

    I solved this problem in like 2 minutes.

  • Trevor Wernisch

    I solved this problem in like 2 minutes.

  • http://agoln.net Logan Buesching

    I’m glad that what I showed you sparked such controversy!

    Also note, if you are just doing it for the purposes of easy-indentation for/in emacs, you can just as easily use whitespace. As you probably know, if you indent the first line of the block, and emacs will just realign the rest of the code to that.

  • http://openradix.org/ Luke Hoersten

    Are you the one that showed me this technique? I thought I got it from Hosking’s code but I could be wrong. If you really did show me this, I will change the post to reflect that. Talk to me in class.

    Also, it’s true that emacs can indent like you’ve mentioned but I like the braces because it’s nice to think of mutual exclusive code as blocks. Good alternative anyway.

  • http://agoln.net Logan Buesching

    Maybe I did… I thought I showed it to someone, but yea, Hosking’s code did that ALL the time. IIRC when I showed you that you said “Yea, that makes sense, Hosking did that in compilers”

  • http://openradix.org/ Luke Hoersten

    That’s right you did show me this. I’ll change that right now. Thanks man.

  • http://agoln.net Logan Buesching

    I’m glad that what I showed you sparked such controversy!

    Also note, if you are just doing it for the purposes of easy-indentation for/in emacs, you can just as easily use whitespace. As you probably know, if you indent the first line of the block, and emacs will just realign the rest of the code to that.

  • http://openradix.org/ Luke Hoersten

    Are you the one that showed me this technique? I thought I got it from Hosking’s code but I could be wrong. If you really did show me this, I will change the post to reflect that. Talk to me in class.

    Also, it’s true that emacs can indent like you’ve mentioned but I like the braces because it’s nice to think of mutual exclusive code as blocks. Good alternative anyway.

  • http://agoln.net Logan Buesching

    someone, but yea, Hosking’s code did that ALL the time. <acronym title=”If I Recall Correctly”>IIRC</acronym> when I showed you that you said “Yea, that makes sense, Hosking did that in compilers”

  • http://openradix.org/ Luke Hoersten

    That’s right you did show me this. I’ll change that right now. Thanks man.