trip logs / gnuvola


Trip Log 2017-09-01 h09

I continue the “indices style upgrade” trip (see accompanying tarball to follow along) now, and delve into patch 2.  Actually, the delving will be much shallower, so this will be much briefer, I promise!  (Of course, there's nothing wrong with being rigorous, like we were for the first patch — up to you.) 

Like patch 1, patch 2 is internal.  It has delta 12 and footprint 30.  Its commit message is:

[tl-mkindex int] Reorg ‘consult-db’.

* sub/tl-mkindex (consult-db fetch-tuples): New proc.
(consult-db cols<-tuples-result): Likewise.
(consult-db): Use ‘fetch-tuples’, ‘cols<-tuples-result’.

All of the locations in the commit message begin with ‘consult-db’.  The first two name procs ‘fetch-tuples’ and ‘cols<-tuples-result’ that are inside ‘consult-db’, while the last names ‘consult-db’ only. 

The first description, “New proc”, is not a proper sentence; it lacks a verb.  The “proc” refers to ‘fetch-tuples’ so the meaning of the description (i.e., expanded and normalized to be a proper sentence) is “Add proc ‘fetch-tuples’ inside proc ‘consult-db’”.  The second description, “Likewise”, means “Identical TEXT as previous description”, so its meaning (after full normalization) is “Add proc ‘cols<-tuples-result’ inside proc ‘consult-db’”.  The last description is the only complete sentence.  It “consumes” what the preceding descriptions “provide”. 

As you can see, the commit message is very stylized and dense in meaning.  To save time henceforth, I'll presume you can do the detailed analysis on your own (if you want), and will highlight interesting features only.  Now for the hunks... 

There are two hunks, the first simply adding a comment on the second.  I'll comment on the comments (hah!) later.  Here's the second hunk, reformatted as a context-diff with line numbers added for reference: 

     1  *** 229,245 ****
     2      (define (consult conn)
     3        (must-sel-q conn query))
     4
     5  !   (let ((cols (map cdr (result->object-alist
     6  !                         (call-with-gnuvola-db-connection consult)
     7  !                         (map type-objectifier
     8  !                              '(text *text
     9  !                                     *text))))))
    10
    11  !     (define (one tag upath title)
    12  !       (cons tag (sort/car< (map cons upath title))))
    13
    14  !     (let-values (((tag upath title) (apply values cols)))
    15  !       (sort/car< (map one tag upath title)))))
    16
    17    (define clean-title                     ; TODO: parameterize
    18      (let* ((cruft "Trip Log ")
    19  - -
    20  --- 234,257 ----
    21      (define (consult conn)
    22        (must-sel-q conn query))
    23
    24  !   (define (fetch-tuples)
    25  !     (call-with-gnuvola-db-connection
    26  !      consult))
    27  !
    28  !   (define (cols<-tuples-result res)
    29  !     (map cdr                            ; discard alist keys
    30  !          (result->object-alist
    31  !           res
    32  !           (map type-objectifier         ; TODO: lift
    33  !                '(text *text
    34  !                       *text)))))
    35
    36  !   (define (one tag upath title)
    37  !     (cons tag (sort/car< (map cons upath title))))
    38
    39  !   (sort/car<
    40  !    (apply map one (cols<-tuples-result
    41  !                    (fetch-tuples)))))
    42
    43    (define clean-title                     ; TODO: parameterize
    44      (let* ((cruft "Trip Log ")

The commit message describes the “what” of the change (as it should), so now I'll describe “why”.  Patch 2 shares the remedial nature with patch 1; I am again clearing out some dubious odors through reduction in (unnecessary) complexity.  (And again, no real genius, here.)  Specifically, the ranklings are two: 

In Scheme, a variable ‘foo’ is an identifier spelled “foo” (U+66, U+6F, U+6F).  Unless it is reserved for use by the language (aka a “keyword”, e.g., ‘if’), it may “name” or “contain” a value, such as a fancy verb (procedure) or the number 42.  For example, in both patch 1 and patch 2, variable ‘consult-db’ names a procedure.  Often, instead of: “variable ‘foo’ names bar”, you see a simplification: “‘foo’ is bar”.  For example, ‘consult-db’ is a proc.  Back to [a], I see: 

My understanding is that ‘cols’ is a local variable [a1] containing a list of some sort [a2].  Furthermore, this value cannot be (and, indeed, is not) modified between assignment and use [a3, a4]. 

My conclusion is that ‘cols’ is gratuitous state and its existence an unseemly departure from this program's path to FP nirvana. 

“Woah ttn, don't get all mystical now!  No extended allegories, metaphors, similes or ilk, please!  You promised to be brief!” 

Well, not entirely true (“briefer” is not identical to “brief”), but anyway thanks for the nudge; I'll revisit the spiritual connection some other time.  What remains for [a] is why did I add ‘cols’ in the first place (i.e., what justification does an experienced programmer have for authoring gratuitous state), and what did I do (in patch 2) about it. 

Well, the answer to the justification question lies in the adjective “experienced”.  Experience has taught me, above all, my fallibility — how easily I make programming mistakes, whether by action, inaction, ignorance, sloth, arrogance, or a combination of these!  I hasten to add that making mistakes is not always negative; personally, I find much of the joy of programming is rooted in adopting a point of view, codifying that pov, and examining the code in action (so to speak).  In short, trial and error. 

So there is a tension here; it's not only easy, but downright fun, to make mistakes.  This is an addictive situation, prone to iterative engagement.  The trick is to know when to stop, to find the right mistake. 

“Not so fast, ttn!  What is a mistake but a wrongness?  So, “right wrongness” — seriously?  You think you can get a job with such confuddlement in your portfolio?!” 

Haha, good point (I think :-D).  For iterative engagement to happen, there must be a relatively stable (and sound) foundation on which different models can be constructed and evaluated.  So, better to say “find the right model”. 

To tie this back to patch 2, my usual MO is to (1) use state to achieve this foundation; (2) iteratively explore models to find the most pleasing fit; and finally (3) squash the model and the foundation together to form the preferred (FP style) shape.  In other words, the foundation is intended to be temporary.  What happened here (prior to patch 2) was that I added ‘cols’ to facilitate printf debugging (step 1), found — by repeatedly writing, evaluating, culling — a suitable set of procs to do what ‘consult-db’ must do (step 2), and then neglected to squash (step 3). 

That's as detailed a recollection as I can muster.  Usually I relish ‘squash-to-FP-style’ as a form of finger meditation (it's a lot of fun in Emacs), so probably there was something more important going on (e.g., pizza) that distracted me.  Dunno; another example of me being human, I suppose... 

All right!  To finish [a], the obvious change to do (and indeed what was done) is to ‘squash-to-FP-style’, which in this case essentially amounts to replacing ‘cols’ with the ‘map’ expression (and restructuring sans ‘let’ form).  This is not directly evident in the after-text because I refactored the ‘map’ expression into (new procs) ‘fetch-tuples’ and ‘cols<-tuples-result’ in the process. 

To clarify, following is an excruciatingly detailed breakdown, AC (atomic change) by AC — six context-diffs in all.  Note that both AC 1 and AC 2 end up in the patch (context-diff lines 24-34): 

;;; Atomic Change 1: Add abstraction: fetch-tuples
*** 229,236 ****
    (define (consult conn)
      (must-sel-q conn query))

    (let ((cols (map cdr (result->object-alist
!                         (call-with-gnuvola-db-connection consult)
                          (map type-objectifier
                               '(text *text
                                      *text))))))
--- 229,240 ----
    (define (consult conn)
      (must-sel-q conn query))

+   (define (fetch-tuples)
+     (call-with-gnuvola-db-connection
+      consult))
+
    (let ((cols (map cdr (result->object-alist
!                         (fetch-tuples)
                          (map type-objectifier
                               '(text *text
                                      *text))))))
                                     *text))))))

;;; Atomic Change 2: Add abstraction: cols<-tuples-result
*** 233,243 ****
      (call-with-gnuvola-db-connection
       consult))

!   (let ((cols (map cdr (result->object-alist
!                         (fetch-tuples)
!                         (map type-objectifier
!                              '(text *text
!                                     *text))))))

      (define (one tag upath title)
        (cons tag (sort/car< (map cons upath title))))
--- 233,247 ----
      (call-with-gnuvola-db-connection
       consult))

!   (define (cols<-tuples-result res)
!     (map cdr                            ; discard alist keys
!          (result->object-alist
!           res
!           (map type-objectifier         ; TODO: lift
!                '(text *text
!                       *text)))))
!
!   (let ((cols (cols<-tuples-result (fetch-tuples))))

      (define (one tag upath title)
        (cons tag (sort/car< (map cons upath title))))

AC 3 and AC 4 “clear the area”.  The “nfc” suffix means “no functional change”.  (Depending on how much you squint, that could be said of all of these changes up to and including patch 2 in its entirety.  Personally, I suffix “nfc” for whitespace and comment munging, only.) 

;;; Atomic Change 3: Move ‘one’ up-scope.
*** 241,251 ****
                 '(text *text
                        *text)))))

!   (let ((cols (cols<-tuples-result (fetch-tuples))))
!
!     (define (one tag upath title)
!       (cons tag (sort/car< (map cons upath title))))

      (let-values (((tag upath title) (apply values cols)))
        (sort/car< (map one tag upath title)))))

--- 241,250 ----
                 '(text *text
                        *text)))))

!   (define (one tag upath title)
!     (cons tag (sort/car< (map cons upath title))))

+   (let ((cols (cols<-tuples-result (fetch-tuples))))
      (let-values (((tag upath title) (apply values cols)))
        (sort/car< (map one tag upath title)))))

;;; Atomic Change 4: Whitespace munging; nfc.
*** 245,251 ****
      (cons tag (sort/car< (map cons upath title))))

    (let ((cols (cols<-tuples-result (fetch-tuples))))
!     (let-values (((tag upath title) (apply values cols)))
        (sort/car< (map one tag upath title)))))

  (define clean-title                     ; TODO: parameterize
--- 245,252 ----
      (cons tag (sort/car< (map cons upath title))))

    (let ((cols (cols<-tuples-result (fetch-tuples))))
!     (let-values (((tag upath title)
!                   (apply values cols)))
        (sort/car< (map one tag upath title)))))

  (define clean-title                     ; TODO: parameterize

AC 5 is the actual replacement (substitution) that fixes [a].  The verb “elide” is arguably incorrect but it sounds so elegant...  :-D 

;;; Atomic Change 5: Elide local var ‘cols’.
*** 244,252 ****
    (define (one tag upath title)
      (cons tag (sort/car< (map cons upath title))))

!   (let ((cols (cols<-tuples-result (fetch-tuples))))
      (let-values (((tag upath title)
!                   (apply values cols)))
        (sort/car< (map one tag upath title)))))

  (define clean-title                     ; TODO: parameterize
--- 244,252 ----
    (define (one tag upath title)
      (cons tag (sort/car< (map cons upath title))))

!   (let ()
      (let-values (((tag upath title)
!                   (apply values (cols<-tuples-result (fetch-tuples)))))
        (sort/car< (map one tag upath title)))))

  (define clean-title                     ; TODO: parameterize

AC 6 is for cleanup.  I intend adjective “degenerate” in the mathematical sense (def. 4), since a scope that does not introduce bindings (i.e., variables and their initial assigned values) has the same meaning to the program as no scope (introduced at that point) at all.  As for “decruft”, that's simply the removal of cruft (obviously :-D). 

;;; Atomic Change 6: Decruft: Remove degenerate scope.
*** 244,253 ****
    (define (one tag upath title)
      (cons tag (sort/car< (map cons upath title))))

!   (let ()
!     (let-values (((tag upath title)
!                   (apply values (cols<-tuples-result (fetch-tuples)))))
!       (sort/car< (map one tag upath title)))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")
--- 244,252 ----
    (define (one tag upath title)
      (cons tag (sort/car< (map cons upath title))))

!   (let-values (((tag upath title)
!                 (apply values (cols<-tuples-result (fetch-tuples)))))
!     (sort/car< (map one tag upath title))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")

At this point, given how much easier it is to understand the change for [a] when presented atomically, you might wonder why didn't I just do these as separate patches?  Sure, the “indices style upgrade” series would be longer, but each delving would be much, much briefer. 

Well, it's a question of balance and audience.  Altogether, AC 1 through AC 6 took approximately ten seconds to perform (in Emacs), and most of that time went towards thinking of some good names for the new procs.  To reify each AC into a proper patch, I would have had to spend at least a minute crafting the commit messages.  That's a disproportionate overhead for next to no immediate benefit (for me).  The truth of the matter is that I did not consider how to present the changes to others in those ten seconds.  Foresight and flow at odds, no surprise... 

OK, I turn to [b] now.  Here is that expression, excerpted from the AC 6 after-text, with one trailing lonely-paren dropped, and line numbers prefixed: 

1  (let-values (((tag upath title)
2                (apply values (cols<-tuples-result (fetch-tuples)))))
3    (sort/car< (map one tag upath title)))

Similar to the ‘let’ form before (AC 5), the ‘let-values’ form introduces a scope.  In this scope, the variables ‘tag’, ‘upath’, and ‘title’ (line 1) are the three separate values (aka the 3-tuple) resulting from the ‘(apply values LS)’ expression (line 2), where LS is the “list of some sort” value (of the former variable ‘cols’).  The three variables are used in the ‘map’ expression (line 3). 

This merits not only the same lament as for [a] (gratuitous state), but an additional cringe-worthy critique: there is no point in structuring LS (via ‘apply values’) only to immediately destructure it (with ‘let-values’).  That's just inane, kind of like splitting the sugar in half and then immediately adding both halves to the banana muffin mix. 

Here are AC 7 through AC 10.  AC 7 through AC 9 clear the area: 

;;; Atomic Change 7: Move ‘sort/car<’ call up-scope.
*** 244,252 ****
    (define (one tag upath title)
      (cons tag (sort/car< (map cons upath title))))

!   (let-values (((tag upath title)
!                 (apply values (cols<-tuples-result (fetch-tuples)))))
!     (sort/car< (map one tag upath title))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")
--- 244,253 ----
    (define (one tag upath title)
      (cons tag (sort/car< (map cons upath title))))

!   (sort/car<
!    (let-values (((tag upath title)
!                  (apply values (cols<-tuples-result (fetch-tuples)))))
!      (map one tag upath title))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")

;;; Atomic Change 8: Whitespace munging; nfc.
*** 246,252 ****

    (sort/car<
     (let-values (((tag upath title)
!                  (apply values (cols<-tuples-result (fetch-tuples)))))
       (map one tag upath title))))

  (define clean-title                     ; TODO: parameterize
--- 246,253 ----

    (sort/car<
     (let-values (((tag upath title)
!                  (apply values (cols<-tuples-result
!                                 (fetch-tuples)))))
       (map one tag upath title))))

  (define clean-title                     ; TODO: parameterize

;;; Atomic Change 9: Move ‘one’ calls up-scope.
*** 245,254 ****
      (cons tag (sort/car< (map cons upath title))))

    (sort/car<
!    (let-values (((tag upath title)
!                  (apply values (cols<-tuples-result
!                                 (fetch-tuples)))))
!      (map one tag upath title))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")
--- 245,254 ----
      (cons tag (sort/car< (map cons upath title))))

    (sort/car<
!    (apply map one (let-values (((tag upath title)
!                                 (apply values (cols<-tuples-result
!                                                (fetch-tuples)))))
!                     (list tag upath title)))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")

AC 10 does the deed.  The after-text appears in patch 2 (context-diff lines 39-41).  The phrase “Use FOO directly” is another way to describe elision, with focus on what remains afterwards instead of what is removed in the process. 

;;; Atomic Change 10: Use ‘cols<-tuples-result’ result directly.
*** 245,254 ****
      (cons tag (sort/car< (map cons upath title))))

    (sort/car<
!    (apply map one (let-values (((tag upath title)
!                                 (apply values (cols<-tuples-result
!                                                (fetch-tuples)))))
!                     (list tag upath title)))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")
--- 245,252 ----
      (cons tag (sort/car< (map cons upath title))))

    (sort/car<
!    (apply map one (cols<-tuples-result
!                    (fetch-tuples)))))

  (define clean-title                     ; TODO: parameterize
    (let* ((cruft "Trip Log ")

Most of these atomic changes are straightforward to understand.  In contrast, understanding AC 7, 9, and 10 is more of a challenge, and explaining (i.e., justifying) them even more so.  To top it off, I haven't even tried to justify the existence of [b] in the first place.  Sigh.  Maybe I'll find the gumption to try later, but for now, it's time to move on.  (In the meantime, trust me or trust me not, how about befriending a programmer and asking for another perspective?) 

I'll finish by briefly (really!) explaining the first hunk (the comment).  It starts by referring to “TODO: lift” (in hunk 2).  The “TODO” is short for “to do”, which is self-explanatory enough, I hope.  I use “lift” in the sense of “hoist” (def. 4).  Overall, when I read it, I hear in my mind: 

“Hey ttn, the result of computing ‘(map type-objectifier '(text *text *text))’ is an unchanging value, so repeating that computation is a shameful abuse of your old computer's CPU, which wastes electricity as well as time, an' you ain't gettin' younger sittn' there pickn' yer nose an' suckn' yer toes yaknow :-/, bla bla bla — why not do it once (somewhere else), save the value into a variable, and refer to that variable here?” 

Can you guess why I hear the comment as question rather than command?  The rest of the first hunk comment refers to the proc compile-outspec from Guile-PG and hints at the “hmmm” that creeps up on most people pondering RDBMS concepts, sooner or later. 

OK, that's all I have to say for patch 2.  So ends the initial preparations — enough (internal) “move”, enough (internal) “reorg”!  Please join me next time for patch 3, the first in the set to effect a user-visible change. 


Copyright (C) 2017 Thien-Thi Nguyen