Org for non-users (2021)
At some point in 2021 I finally understood why I have always found Org Mode so hard to use. To make a long story short, I am a very bad user: when I try to use something and I can't make a clear mental model of what's going on my first reaction is to try to look at the data structures and at the source code - and the code that implements code blocks in Org is hard to understand.
TL;DR: I am a "non-user" and I found a way to document Org using eev that works for me and that may also work for other non-users like me, and I recorded a video explaining and demo-ing all this. If you like to watch videos with subtitles in high speed, then run the two "wget"s below to download a local copy and the "mpv" to play it (hint: use these keys):
Lots of important-ish things happened in november and december/2021:
In Org: Karl Voit gave this presentation on OrgDown at the EmacsConf2021; it was very well received at the conference and very badly received on reddit. He wrote this blog post about that. There were huge threads on emacs-orgmode about this: 1, 2. I learned a lot from these threads, and two of my favorite e-mails were this one by Russell Adams and this one by Ihor Radchenko.
In eev: I gave this presentation at the EmacsConf2021 and this workshop. I finally understood how alien one of the principles of eev - "save elisp hyperlinks to everything interesting that you find" - sounds to most people. I started to work on this tutorial on saving links. Sachac and grym helped me on #emacs with some of my main doubts about code blocks. I recorded the video below, created this page, and sent this e-mail to the Org mailing list announcing the page and the video. Samuel Banya sent this e-mail to the list mentioning Rainer König's videos and I answered him saying that 1) here is how I used eev to download and index some of Rainer König's videos, and that 2) I don't know how to do something similar to those indexes in Org yet.
TODO: write more.
To watch the video click on the thumbnail below.
The .org file that I used in the video is here.
For more info on the video - including an index in elisp - click here.
I got the photo of the Winter Moon Mask from this group: Northwest Coast Native Carvers.
The instagram page of the artist is here, and
his site is here.
Index of the video (elisp here):
0:00
0:12 To make two long stories short
0:18 In 2021 I finally understood
0:26 the documentation is written for users
0:36 I am a non-user
0:42 I can use eev to write the docs that I need
0:57 I also found something really weird
1:03 The main intent of eev ... take executable notes
1:11 one part that everyone understands
1:23 and a part that is much harder to learn
1:54 I'm working on a way to make this easier
2:04 This is the initial page of the tutorial
2:17 Let me explain what are users and non-users
2:51 a non-user
3:04 questions like: how does Org handle code blocks
3:20 repeatable tests
3:35 An example
3:49 This thing here is a source block
4:00 we can run the source block with C-c C-c
4:09 if I do that Org creates a results block
4:25 C-c C-c executes org-babel-execute-src-block
4:40 I can learn more with M-h M-k C-c C-c
4:55 I get this temporary buffer
5:04 in these notes I selected some hyperlinks
5:23 (find-efunctiondescr ...) shows the description
5:29 it's quite long
5:39 the last one says: in a code block...
5:49 (set org-confirm-babel-evaluate nil)
6:02 org-babel-remove-result-one-or-many
6:17 M-h M-k C-c C-v k
6:38 (find-efunctiondescr ...) shows the description
6:49 (find-efunction ...) shows the source code
7:13 I also wanted to understand the data structures
7:48 I asked on IRC ... (org-element-context)
7:58 if I execute it here ... the result is big
8:05 so let me use find-epp
8:18 the important part is in the cadr
8:34 and I can use find-eppp
8:43 I can see the keywords and their values
8:49 suppose that I want to undestand this :begin
8:54 (setq p (cadr (org-element-context)))
9:08 p now holds a property list
9:26 I can use the function of eev that highlights
9:40 when I run I see that the region includes
10:07 Another thing:
10:17 we can create a src block by hand
10:24 there's a way to type it more quickly
10:28 (find-orgnode \"Structure templates\")
10:35 If I type C-c C-,
10:46 `s' is the letter that I want
10:50 and if I type C-c C-, s
10:53 (there's some help here)
11:00 and I get a source block
11:12 of course I want to know how
11:20 I used M-h M-k C-c C-,
11:48 I selected these two sexps
11:59 the target of this sexp is the description
12:16 the target of this sexp is the source code
12:21 I can use find-efunctiondescr and click on
12:44 (find-evardescr 'org-structure-template-alist)
12:55 we can get help on that variable by typing M-h M-v
12:59 we get a buffer with elisp hyperlinks
13:15 so one of them just opens the description
13:30 (describe-variable '...) has a big result
13:42 so I prefer to use this variant
13:49 with this I can get a description
14:03 here it shows the current value
14:10 refine it with a \"src\"
14:26 so this link ... and searches for
14:40 to go to the defcustom
14:52 here is its initial value
15:05 this (eek \"C-c C-, s\") is a reminder
15:26 and this is an example of a source block
15:39 Just one thing more
15:45 by default when I type M-e
15:51 it opens the target in the same window
16:03 but if I type M-2 M-e or M-3 M-e
16:17 this is practical because I can show
The index for this this video is also here,
and its subtitles in Lua are here.
The rest of this page contains a conversion of the subtitles in Lua
to a slightly more readable format.
00:00 Hi! My name is Eduardo Ochs...
00:03 I'm the author of an Emacs package called
00:06 eev, and the title of this video is
00:09 "Org for non-users". Let me explain what
00:11 I mean by this.
00:13 To make a long story short, or
00:15 actually to make two long stories short...
00:18 in 2021 I finally understood why I've
00:22 always found Org mode so hard to learn.
00:25 And it turns out that the documentation
00:28 of Org is written for "users",
00:31 and I realized that I'm a very bad "user".
00:34 I'm going to define this thing soon.
00:36 I'm a "non-user" in a sense that I'm
00:39 also going to define soon.
00:42 i also found that i can use eev -
00:45 my package - to write the documentation for
00:47 Org that I need...
00:49 and the technique
00:51 that i'm going to explain here _may_ be
00:53 useful to other non-users like me.
00:57 I also found something that I found
01:00 really weird...
01:02 let me let me explain. The main intent
01:04 of eev is to let you "take executable
01:07 notes of everything that you do"
01:10 and "taking executable notes" has one
01:13 part that everyone understands, which is
01:16 to "save all commands that you send to
01:18 external programs" - usually REPLs -
01:22 and it has a part that is much harder to
01:26 learn, which is to
01:28 "save elisp hyperlinks to everything
01:31 interesting that you find"...
01:34 and I discovered that this part is
01:36 much harder than than I thought...
01:39 it was totally natural for me when I
01:41 started using Emacs, I started doing that
01:43 immediately...
01:45 but I discovered that most people can't
01:47 even imagine how is the workflow of
01:49 someone who works
01:52 like this, and so i'm working on a way to
01:56 make this easier to understand...
01:59 I'm writing a tutorial about this - it's
02:01 called `(find-saving-links-intro)' -
02:03 let me show just the initial page
02:06 of this tutorial... it starts
02:09 with a section named
02:11 "Reading diagrams aloud", but
02:13 this is not important to us now...
02:17 so now let me explain what are "users"
02:20 and "non-users",
02:23 and let me use one feature of
02:26 Org mode, called "code blocks",
02:28 or "source blocks", as an example.
02:33 If a "user" wants to learn how to use code
02:35 blocks in Org then the "user" is probably
02:38 going to need to learn what are the key
02:40 sequences, what are the main examples -
02:44 or some examples - and the "user"
02:46 does not want to see the source code...
02:51 and a "non-user" is going to think in a
02:53 way that is very different. A "non-user"
02:56 needs to have a clear mental image of
02:58 what is going on behind the scenes...
03:01 so a "non-user" is going to ask questions
03:05 like: how does Org handle code blocks
03:08 _internally_?
03:10 What are the data structures that it
03:12 uses?
03:14 What is the source code like? What are
03:16 the functions? How is it structured?
03:20 And a "non-user" is also going to need
03:24 repeatable tests, that can be
03:26 executed either unchanged or with little
03:29 changes, to check what is going on.
03:34 So let me start with an example...
03:41 this thing here...
03:46 no, let me use a smaller font here...
03:49 this thing here is a source block
03:52 containing shell code. Actually it
03:54 contains lots of comments and
03:56 just this line here of non-trivial shell
03:58 code... and we can run a code block by
04:02 putting the cursor - the point - inside
04:05 the code block and typing `C-c C-c'.
04:09 If I do that
04:11 what happens is that Org creates
04:14 a second block here called "results"
04:17 with the result of this command here
04:18 formatted in a certain way, as columns...
04:21 Let me undo this.
04:26 It turns out that typing `C-c C-c'
04:29 executes this function here,
04:32 `org-babel-execute-src-block'...
04:34 so executing this sexp is equivalent to
04:37 typing `C-c C-c'...
04:39 (undo again)
04:41 and I can learn more about `C-c C-c'
04:45 in Org mode by running this key sequence
04:47 here: `M-h M-k', which is
04:51 a key sequence for help on keys on eev...
04:54 and if I run this
04:58 I get this temporary buffer with lots of
05:01 elisp hyperlinks...
05:03 and in these notes here I've selected
05:05 some of these elisp hyperlinks -
05:08 these two -
05:09 and I've just indented them a bit.
05:14 So I took these two hyperlinks here...
05:19 and copied to my notes and
05:21 indented them.
05:23 This one
05:24 shows the description of this function,
05:27 which is this...
05:29 the description is quite long because
05:31 the effects of this function depends on
05:33 the context...
05:36 one of these things here -
05:40 the last one - says: if the cursor is on a
05:43 code block, evaluate it.
05:45 And blah blah blah blah blah blah...
05:49 I had to set this thing here to nil
05:52 to make Org
05:54 stop asking me every time if I want
05:57 to evaluate this thing or not...
06:01 and Org also has a key sequence for
06:08 removing the results block
06:11 this key sequence is `C-c C-v k'.
06:16 I can get a lot of help about...
06:18 sorry, lots of elisp hyperlinks about
06:20 this key sequence by running this sexp
06:22 here... and I've also
06:26 selected these ones...
06:29 sorry, these ones here -
06:32 and copied them to the other buffer, and
06:34 indented them a bit...
06:39 so this sexp here
06:44 shows a description of this function
06:48 and this sexp here shows the source code
06:51 of this function - it is here.
06:54 So by executing this I can
06:58 see what are the names of the functions
07:00 that do most of the work - because this
07:02 function is quite simple, so it
07:07 essentially calls other functions to do
07:11 practically everything.
07:14 And I also wanted to understand
07:16 what the data structures associated
07:20 to a code block, because I knew that
07:23 when Org did use the code block
07:25 it had to be done in two parts...
07:28 one part was parsing
07:29 the code block and
07:31 representing it internally in a certain
07:33 way,
07:34 that I imagined that would include
07:38 information about this,
07:40 information about the beginning and
07:42 the end of the
07:44 of the code block in this buffer,
07:46 and so on...
07:48 so I asked on IRC what is the function
07:51 that does this parsing, and people
07:53 pointed me to a function called
07:56 `org-element-context',
07:58 and if I execute it here
08:01 I get a result that is too big - it
08:03 doesn't fit very well in the echo area.
08:06 So let me use one of my favorite
08:08 pretty-printing functions
08:10 to take a look at this...
08:14 it's not very pretty.
08:16 So... I can see
08:17 that the important part is here,
08:20 so I have to discard this
08:22 initial part...
08:24 if I take the cadr of
08:28 the result of this I get the part that is
08:30 most interesting - this one here...
08:32 and I can use this variant
08:35 of my favorite pretty printing function
08:37 to print it in a better way.
08:40 So, if I execute this I can see
08:44 the keywords and their values...
08:48 and, for example, suppose that I want to
08:51 understand what are these numbers here...
08:55 one thing that I can do is that I can...
08:59 well, let me take the result of this and
09:03 and put it in a variable called p...
09:07 and p now holds a property list
09:10 and I can use plist-get to
09:13 get the value of the field :begin,
09:17 and this other plist-get here to get
09:22 the value of the field :end...
09:26 and I can use the function of eev
09:29 that does these highlightings
09:31 to highlight what is the region between
09:34 the value of the field :begin
09:37 and the value of the field :end,
09:41 and when I run this I see something
09:43 funny...
09:46 this region includes these blank
09:49 lines here.
09:51 So:
09:54 I can use this thing here to understand
09:56 the data structures, and
09:58 then I can find some
10:02 non-trivial information here...
10:08 I'm just going to show another thing
10:10 that can be done with source blocks,
10:12 and what I discovered
10:14 about the source code, which is...
10:16 we can create a source block
10:19 by hand, by typing this #+BEGIN thing
10:21 and this #+END thing by hand,
10:23 but there's
10:26 a way to type that more quickly.
10:29 It is described in this node of the
10:32 Org manual, called "Structure Templates",
10:35 and it explains that
10:37 if I type `C-c C-,'
10:41 and then one of these letters here
10:44 I get these blocks...
10:46 so `s' is the one that I want,
10:50 and if I type `C-c C-,'...
10:55 there's some help here -
10:57 so I can type `s',
11:00 and I get a source block.
11:04 Here i can put "shell" or "python"
11:06 or wherever, any supported language...
11:11 and
11:12 of course I want to know how this is
11:14 implemented, how this can be extended to
11:17 other languages, and so on...
11:19 so the first thing that I did was that I
11:22 used my key sequence for
11:25 getting help on key sequences,
11:28 which is `M-h M-k',
11:32 I used it to get help on `C-c C-,'...
11:36 let me try it here... `M-h M-k C-c C-,'
11:45 I get this thing here -
11:47 and I selected
11:49 these two sexps, and copied them to
11:52 my notes - I got these ones here...
11:55 and so this one points to...
11:59 the target of this sexp here is the
12:01 description of this function,
12:03 which is this thing here...
12:06 "Insert a block structure of the
12:08 type blah blah blah
12:09 #+begin_foo/#+end_foo"...
12:11 and then it
12:13 says some more technical things here...
12:16 and I can use this sexp here to go
12:19 straight to its source code...
12:22 I mean, I can use this, and I can
12:25 click in this button here, but I prefer
12:28 to have a link that points straight to
12:30 the source code, and it's this one...
12:34 and so I can read these things here to
12:36 understand what is going on,
12:40 which variables it uses, and so on...
12:43 and it turns out that this thing uses a
12:46 variable called
12:48 `org-structure-template-alist',
12:51 and if I...
12:54 I can get help on this variable
12:57 by typing `M-h M-v' - I get a buffer
13:00 with elisp hyperlinks, like this...
13:02 and what I did was that I generated a
13:05 buffer like this, and selected some of
13:07 the most interesting hyperlinks, and
13:09 copied them to my notes... here -
13:15 so one of them is this `find-evardescr',
13:18 that
13:20 opens the description of this variable...
13:24 just let me show that
13:26 it appears here... usually people
13:31 would use `describe-variable',
13:33 but when we execute it as a sexp
13:36 it returns a big return value, here, that
13:40 messes up my screen... so I prefer to use
13:43 this variant here - `find-evardescr'.
13:49 So, yeah -
13:50 with this I can get a description of this
13:55 variable
13:57 that's a bit more technical than the
13:58 description in the
14:00 manual of Org...
14:03 and here it shows the current value of
14:06 this variable.
14:10 I can use...
14:12 actually when I was looking at this I
14:14 discovered that
14:18 there was something interesting here, and
14:19 I can could point to this something
14:21 interesting by refining this hyperlink
14:24 and making it search for this string
14:26 in the description...
14:28 so this link
14:31 opens this description of the variable
14:33 and searches for the first occurrence of
14:35 the string "src"...
14:39 and
14:42 I can also run this sexp here to go to
14:45 the place in the source code
14:48 in which this variable is defined... it's
14:50 defined with a `defcustom'... here
14:53 is its
14:54 initial value,
14:57 here is its description, here's how it
14:59 can be customized, and so on...
15:03 and
15:06 this is just a reminder that I can create
15:09 source blocks by typing something
15:12 like this... I think that I've used that
15:13 before -
15:16 if I type `C-c C-, s'
15:21 I get a
15:22 source block that can I edit by hand, and
15:26 this is an example of a source block
15:28 created in that way, with some
15:32 non-trivial shell commands here, inside...
15:38 Yeah, that's it - that's what I
15:40 wanted to show. Oh, no, no - just one thing
15:42 more...
15:45 by default when I type `M-e' on
15:49 a sexp the `M-e' opens the target
15:52 of the sexp in the same window...
15:55 so if I execute this
15:57 I open the description of this
16:00 function in the same window, but if I
16:03 execute it... if execute `M-e' with
16:07 the prefix 3, I mean, if I type `M-3 M-e',
16:11 or `M-2 M-e', I display
16:14 the target in the right window,
16:16 and this is practical because then I can
16:19 show all these targets
16:21 very quickly - here.
16:29 Yeah! So, that's it...
16:31 that's what I wanted to show in this
16:33 video. Bye! =)