[INCLUDE TH/speedbar.blogme]
[# SETFAVICON IMAGES/forth.png]
[# SETFAVICON IMAGES/forthlua.png]
[# SETFAVICON IMAGES/forthluawhite.png]
[SETFAVICON IMAGES/forthsun.png]
[# SETFAVICON http://anggtwu.net/eev-current/eev-icon.png]
[#
(defun c () (interactive) (find-blogme3-sh0-if "davinci"))
;; http://angg.twu.net/davinci.html
;; file:///home/edrx/TH/L/davinci.html
#]
[# (defun c () (interactive)
(let ((output (find-sh0 "cd ~/TH/L/;
cp ~/TH/davinci.blogme ~/TH/L/TH/davinci.blogme;
lua51 ~/blogme3/blogme3.lua \
-o davinci.html -i TH/davinci.blogme")))
(if (equal output "") (message "ok") (find-estring output))))
;; (find-blogmefile "README")
;; (find-blogmefile "INTERNALS")
;; (find-blogmefile "")
;; file:///home/edrx/TH/L/davinci.html
;; http://angg.twu.net/davinci.html
#]
[lua: L = R; LR = R ]
[lua: def [[ br 1 _ "
" ]] ]
[lua:
load_TARGETS()
def [[ __ 2 str,text _target[str] and HREF(_target[str], nilify(text) or str)
or BG("red", str) ]]
def [[ _ 1 body __(gsub(body, " ", "."), body) ]]
]
[_TARGETS
IUP => http://www.tecgraf.puc-rio.br/iup/
CD => http://www.tecgraf.puc-rio.br/cd/
IM => http://www.tecgraf.puc-rio.br/im/
LuaCmd => http://www.tecgraf.puc-rio.br/luacmd/
Emacs => emacs.html
Lua => luaforth.html
CNPq => http://www.cnpq.br/
TecGraf => http://www.tecgraf.puc-rio.br/im/
Tcl/Tk => tcltk.html
lua50implpaper => http://www.tecgraf.puc-rio.br/~lhf/ftp/doc/jucs05.pdf
]
[_TARGETS
scintilla -> (find-es "scintilla")
Scintilla -> (find-es "scintilla")
scite -> (find-es "scintilla")
SciTe -> (find-es "scintilla")
davinci.e -> (find-es "davinci")
tcmd.tcl -> (find-angg "DAVINCI/tcmd.tcl")
tcmd-sshot -> (find-anggfile "DAVINCI/tcmd-1.png")
luacmd/debug.gif => http://www.tecgraf.puc-rio.br/luacmd/debug.gif
luacmd => http://www.tecgraf.puc-rio.br/luacmd/
]
[lua: -- (find-eleimfile "quail/sgml-input.el" "‘")
def [[ ` 1 body "‘$body’" ]]
def [[ `` 1 body "“$body”" ]]
-- (find-anggfile "TH/blogme3.blogme")
def [[ MIMG 1 semiurl IMG("http://angg.twu.net/"..semiurl, "alt") ]]
def [[ MIMG 1 semiurl IMG(""..semiurl, "alt") ]]
def [[ LISTING 1 semiurl R("$semiurl.html", "http://angg.twu.net/$semiurl") ]]
def [[ STANDOUT 1 body BGFG("red", "yellow", body) ]]
loada2html()
def [[ HTMLIZELINES 1 body htmlizelines(body) ]]
def [[ E' 1Q body htmlizelines(body) ]]
code_c_d_remote("luacmdhtml", "http://www.tecgraf.puc-rio.br/luacmd/")
]
[lua:
loadlpeg()
twordtable = {}
-- translateword = function (str) return string.rep("@", strlen(str)) end
translateword = function (word)
if twordtable[word] then
return HREF(twordtable[word], word)
else
return word
end
end
Pos = lpeg.Cp()
AnyChar = lpeg.P(1)
WordChar = lpeg.R("AZ", "az", "09") + lpeg.S("/")
Word = WordChar^1
OtherWord = lpeg.S(" .") * lpeg.S("-")^-1 * Word
Words = Word * OtherWord^0
WordsT = Words / translateword
SbeChars = Pos * AnyChar^0 * Pos
SbeCharsThen = function (patT)
return Pos * lpeg.P { Pos * patT + AnyChar * lpeg.V(1) }
end
SbeWordsT = SbeCharsThen(WordsT)^0 * SbeChars
sbematch = function (SbePat, str, f)
return sbeconcat(str, f)(SbePat:Ct():match(str))
end
lpeg.sbematch = sbematch
def [[ TRANS 1 str SbeWordsT:sbematch(str, Q) ]]
]
[#
«.overview» (to "overview")
«.iup,cd,im» (to "iup,cd,im")
«.davinci-0.0.1» (to "davinci-0.0.1")
«.long-way» (to "long-way")
«.short-way» (to "short-way")
«.SIGUSR2» (to "SIGUSR2")
«.peek.lua» (to "peek.lua")
«.gdb» (to "gdb")
#]
[htmlize [J Edrx's page on DaVinci (an IDE for Lua)]
[P [STANDOUT [BF Ssshhh!!!]] The DaVinci project has not been
announced officially yet! [BR] Please be discreet about this page, and
be aware that everything here may change... [BR] Thanks!]
[P [STANDOUT [BF Disclaimers:]]]
[P (1): I haven't done all my homework on (learning) Scite yet - I
know very little about it at this moment, and I haven't even followed
the tutorials at the Lua wiki thoroughly - I had to do other things
first.]
[P (2): (this page is a rat's nest - [R contact.html get in touch])]
[P (3): I'm temporarily working only on [R iup.html IupScintilla] - it
needs to be finished urgently.]
[# br]
[#
[P 2007aug24: [BF I'm in holidays] (I decided it).
[BR] I haven't received my first payment yet.
[BR] It was due on 2007aug07.
[BR] During the last two weeks I was always
[BR] expecting that the money would be deposited
[BR] in my account "in one or two days".
[BR] Financial hell. Productivity dropped to zero.]
[P 2007aug27: Problems solved. Work resumed.]
#]
[P (Last update: 2007oct08)]
[RULE ----------------------------------------]
[WITHINDEX
[RULE ----------------------------------------]
[P DaVinci will be an IDE for Lua based on SciTe/Scintilla and IUP. It
will be composed of:]
[LIST2
[J a source code editor. This will initially be a slightly modified
SciTe - SciTe is already scriptable in Lua, and can do syntax
highlighting on Lua code - but in some months we also want to be
able to control Scintilla directly from Lua, and to build other
editors based on Scintilla, without SciTe.]
[J A debugger. This means running a Lua debugger (either ldb or
remdebug, or both) from inside the editor, and having a separate
window (like this screenshot) showing the current line being
executed in the source code, breakpoints, values of some
variables, etc. This separate window will be controlled by another
graphical library - in early prototypes Tk, and later IUP.]
[J A "project manager" - maybe something like this screenshot. It
should be able to produce a single executable file]
]
[P Side-products:]
[# http://en.wikipedia.org/wiki/Tcl/tk]
[RULE ----------------------------------------]
[P Main news (details soon): I've been hired! An official
announcement is due [BF tonight] (thursday, 2007jul19). We have some
trivial running code - [AL DAVINCI/intro.lua intro.lua] (incomplete) -
and sketches of lots of lots of what is going to be done.]
[P (2007jul25): I've been postponing the official anouncement 8-\... I
was afraid that people would think that the code here is too trivial
and they wouldn't see where this is heading to, so I decided to add
more examples to [AL DAVINCI/intro.lua.html intro.lua] before the
release - ideally, I would put more examples of controlling SciTe and
Scintilla from Lua, and examples of starting Tcl/Tk and IUP from Lua
too - but then I discovered a [ES davinci#wish-event-loop problem with
the event loop] in [ES davinci#libiuplua51.so Lua+libiuplua51] - and I
need to explain that clearly before releasing the announcement...]
[RULE ----------------------------------------]
[sec «overview» (to ".overview")
H3 1. Overview]
[P Some people at [R http://www.tecgraf.puc-rio.br/ TecGraf] are
trying to write an IDE for [HREF luaforth.html Lua] based on [R
http://www.tecgraf.puc-rio.br/iup/ IUP] and [R
http://www.scintilla.org/ Scintilla], and I'm trying to join them.]
[P My e-scripts about DaVinci are [ES davinci here].]
[# (find-angg "DAVINCI/tcmd.tcl")
#]
[P 2007jul04: [__ tcmd.tcl TCmd]
(screenshot [__ tcmd-sshot here]).
This is a prototype code for something like the
[__ luacmd/debug.gif debug window] of
[__ luacmd LuaCmd].
The [__ tcmd.tcl source] explains the rationale and inner workings.]
[P [SCALEDIMAGE 25% DAVINCI/tcmd-1.png]]
[RULE ----------------------------------------]
[H3 The DaVinci project: what we are trying to do]
[P (2007jul08)]
[P My first impression from reading the official specifications of the
DaVinci project was that what we are trying to build is an IDE for
Lua: that is, an editor, a debugger, and a "project manager" that
keeps a list of source files, lets us edit any of them, and knows how
to produce a single executable file from them all. All of this must be
scriptable in Lua, of course...]
[P The Department where Lua was created - [_ TecGraf] - also has other
groups besides the Lua group, working on projects unrelated to Lua.
Several of these projects are libraries that can be used directly from
C; they do have Lua bindings, but they're independent from Lua.]
[P The people who started the DaVinci project have created several of
these graphical libraries - for example, [_ IUP], [_ CD], and [_ IM] -
and a previous incarnation of DaVinci: [_ LuaCmd]. My impression as an
outsider was that no one without direct contact with TecGraf was using
those libraries; well, from [__ LuaCmd this page] we can see that the
source code for some of these libraries has only become publically
available a few years ago...]
[P So, we will try to integrate DaVinci - the IDE - with these
graphical libraries in several ways. The DaVinci program itself will
be built on top of Scintilla (initially Scintilla and Scite), but
parts of it - the "debug window", for example - will use IUP]
[RULE ----------------------------------------]
[sec «iup,cd,im» (to ".iup,cd,im")
H3 2. IUP/CD/IM]
[P One thing that made both TecGraf and a [__ CNPq funding agency] more
interested in the DaVinci project was that it involves IUP.]
[P The department where Lua was created - [_ TecGraf] - also has other
groups besides the Lua group, working on projects unrelated to Lua.
Some of these projects are libraries that can be used directly from C;
they do have Lua bindings, but they are independent from Lua.]
[P [_ IUP], [_ CD] and [_ IM] are some of these libraries.]
[P I used to have the impression that only people with close ties to
TecGraf used IUP and CD and IM. Some years ago when I took a look at
them their source code wasn't even publically available ([IT confirm;
see [R [TGT (find-luacmdhtmlw3m "prod.html")] this]]), and I've tried
to compile their current source packages recently and found that very
hard. But a quick look at the LuaForge pages reveals that they get
lots of downloads, so they can't be as "outsider-unfriendly" as I
thought they were... anyway: one of my duties in the DaVinci project,
as I am an outsider who thought that IUP and CD and IM were hard to
use (because they are hard to compile, etc etc), is to fix these
defects and make them "outsider-friendly" for more kinds of
outsiders.]
[IT (To do: add links to what I already have - which is not much. [R
[TGT (find-es "davinci" "iup-binaries")] This e-script] and some
things below it are what I use to unpack the precompiled ".so"s and
load them into a running Lua interpreter; I didn't go very far in my
attempts to recompile from the sources).]
[P The DaVinci project will use IUP in several ways.]
[LIST2
[J Some of our examples of running Lua programs from SciTe will be
of programs that link with IUP and CD and IM.]
[J [_ lcmd] will use IUP for its GUI.]
[J (debugging Lua programs that link to IUP)]
[J At some point we want to be able to use Scintilla as a text
widget for IUP; the versions of DaVinci that will use that will be
much easier to port than the other ones.]
]
[RULE ----------------------------------------]
[sec «davinci-0.0.1» (to ".davinci-0.0.1")
H3 3. What we have now: DaVinci 0.0.1]
[# lua: --
print("hello")
]
[P Status: version 0.0.1 is ready! It is just a proof of concept, it
doesn't look like an IDE at all, and it will run only on *nix-likes
(for reasons that we will see below) - but it has some nice running
code! Take a look at the diagram below:]
[lua:
twordtable = {
["emacs"] = "emacs.html",
["tcmd.tcl"] = "DAVINCI/tcmd.tcl.html",
["shell"] = TGT '(find-enode "Interactive Shell")',
["intro.lua"] = "DAVINCI/intro.lua.html",
["lua50init.lua"] = "LUA/lua50init.lua.html",
[".SciTEUser.properties"] = ".SciTEUser.properties.html",
["luatclbridge.so"] = "LUA/luatclbridge.c.html",
-- (find-angg "LUA/luatclbridge.c")
["libiuplua5.1.so"] = "e/davinci.e.html#libiuplua51.so",
-- (find-es "davinci" "libiuplua51.so")
}
]
[PREBOXBLOGME [TRANS ['
.________________
| |
| emacs |
| :: |
| :: |
| ____\/_______|__ ___________________
| | | | |
| | shell ::::::::::::::::::::::::::::::::::> | |
| | :: | | tcmd.tcl |
|__|_____::_________| | (Tcl/Tk/Expect) |
:: ^ | |
:: : stdout | |
________\/_____:__ | |
| : | | |
| SciTe 1.74 : | | |
| :: : | | |
| ______\/____:__| | |
| | | <--- /tmp/ee.tcmd.pid <--- | |
| | Lua 5.1 | ---> /tmp/ee.tcmd.tcl | |
| | | ------- kill -USR2 ------> | |
|__|_______________| /tmp/ee.tcmd.tcl ---> |___________________|
^ ^
: :
intro.lua :
:
lua50init.lua
]]]
[P It corresponds to [HREF DAVINCI/davinci-0.0.1.png this
screenshot]:]
[# IMG DAVINCI/scite-lua-dostring-1.png]
[# (find-sh0 "cp -v ~/DAVINCI/davinci-0.0.1.png ~/TH/L/DAVINCI/")
]
[P [SCALEDIMAGE 25% DAVINCI/davinci-0.0.1.png]]
[RULE ----------------------------------------]
[sec «long-way» (to ".long-way")
H3 4. The long way: select.lua and lcmd.lua, in and outside Scite]
[P Here is what we expect to have running very soon: the Lua
interpreter inside SciTe being able to the load IUP (and IM and CD) as
libraries, and also Tcl/Tk in the same way; this Lua interpreter
inside SciTe already has bindings to call functions from SciTe and
Scintilla. On top of that it won't be hard to add a Lua debugger (ldb
or remdebug), changing its current textual interfaces to a [_
LuaCmd]-like interface using a IUP or a Tk window. A diagram:]
[PREBOXBLOGME [TRANS ['
.__________________
| | |
| Scintilla | |
| /\ | |
|_______::____| | ________________
| :: | | | |
| SciTe 1.74 | | | Tk |
| :: | | |_______|
| ________\/_____| ___________|_____ /\ |
| | | | | :: |
| | Lua 5.1 ::::>| luatclbridge.so ::::> Tcl |
| | | |_________________|__________|
| | | _________________ __________
| | | | | |
| | ::::>| libiuplua5.1.so ::::> IUP |
| | | |_________________| CD |
| | | | IM |
|__|_______________| |________________|
]]]
[P But how would the event loop of this work? We would have windows of
three different types - Scintilla, Tk, IUP - and we'd need to handle
events coming from any one of them... the event handlers are
different, and the event handler in SciTe will have to deal with two
other event handlers that are "on the other side of the Lua
layer"...]
[P One preliminary step is to implement this:]
[PREBOXBLOGME [TRANS ['
. ________________
| | |
| | Tk |
| |_______|
_______________ ___________|_____ /\ |
| | | | :: |
| ::::>| luatclbridge.so ::::> Tcl |
| Lua 5.1 | |_________________|__________|
| (lcmd 0.2) | _________________ __________
| | | | |
| ::::>| libiuplua5.1.so ::::> IUP |
| | |_________________| CD |
| | | IM |
|_______________| |________________|
]]]
[P Here the event handler will have to deal with events from two kinds
of windows - Tk and IUP. Also, if we start a Lua interpreter in
interactive mode and then we load luatclbridge.so and libiuplua5.1.so
we don't want to lose the interactive prompt; so there's
stdin/readline too, and, why not, let's add on top of that a signal
handler, so that this Lua interpreter will also be able to react to,
say, SIGUSR2s.]
[_TARGETS
#tcl => http://wiki.tcl.tk/2221
]
[P The Tcl people are always doing things like that (and there's the
[_ #tcl] IRC channel at freenode!), so I will
start by implementing this ([NAME lcmd-0.1 [BF lcmd 0.1]]):]
[PREBOXBLOGME [TRANS ['
. ________________
| | |
| | Tk |
| |_______|
_______________ ___________|_____ /\ |
| | | | :: |
| ::::>| luatclbridge.so ::::> Tcl |
| Lua 5.1 | |_________________|__________|
| (lcmd 0.1) |
| |
| |
| |
| |
|_______________|
]]]
[P I expect_[BF ed]_ to have this running by [BF August 16] - but the
priorities have changed ([R iup.html IupScintilla] has to be finished
soon).]
[RULE ----------------------------------------]
[sec «short-way» (to ".short-way")
H3 5. The short way: IUP using Scintilla as a text widget]
[P At some point - by november, I hope - we should also have this:]
[PREBOXBLOGME [TRANS ['
._____________......
| | :
| Scintilla |<:::::::::::::::::::::::::::::::::.
| | : ::
|_____________| : ::
: : ::
: : ::
: : ::
: _______________: ::
: | | ::
: | Lua 5.1 | ::
: | | ::
: | | _________________ _____::___
: | | | | |
: | ::::>| libiuplua5.1.so ::::> IUP |
: | | |_________________| CD |
: | | | IM |
:..|_______________| |________________|
]]]
[P here we will have a direct connection - in C, and independent from
Lua, but with Lua bindings - between IUP and Scintilla; the idea is
that IUP 3.0, that has not been released yet, should be able to use
Scintilla as a text widget. And on top of that version of IUP we will
implement a version of DaVinci that will not use SciTe.]
[P This will be more portable and easier to install than the other
implementations of DaVinci, but it is much less hacker-friendly than
the other ideas... I will start with the other approaches, then
produce prototypes in Lua for this one, then rewrite these prototypes
in C.]
[RULE ----------------------------------------]
[sec «SIGUSR2» (to ".SIGUSR2")
H3 6. Sending commands with SIGUSR2]
[P The lower part of the diagram that describes DaVinci 0.0.1, above,
shows a BIIIG arrow going rightwards; it is made of four sub-arrows,
the uppermost of them going "in the wrong direction", from tcmd.tcl to
Lua, while the other three go from Lua to tcmd.tcl. That big arrow -
reproduced below - shows how tcmd.tcl receives commands from the
outside:]
[PREBOXBLOGME [TRANS ['
. ___________________
| |
| |
_______________ | tcmd.tcl |
| | <--- /tmp/ee.tcmd.pid <--- | (Tcl/Tk/Expect) |
| Lua 5.1 | ---> /tmp/ee.tcmd.tcl | |
| | ------- kill -USR2 ------> | |
|_______________| /tmp/ee.tcmd.tcl ---> |___________________|
]]]
[P tcmd.tcl saves its pid to [TT /tmp/ee.tcmd.pid], and then it waits
for [TT SIGUSR2]s; to send Tcl commands to it we write these commands
into [TT /tmp/ee.tcmd.tcl], then we send a [TT SIGUSR2] signal to
tcmd.tcl; then tcmd.tcl wakes up, reads the commands in [TT
/tmp/ee.tcmd.tcl], and executes them.]
[P The code to listen for [TT SIGUSR2]s in tcmd.tcl is just this,]
[PREBOXBLOGME ['
package require Expect; # "trap" and "pid" are Expect commands
proc pidfile {} { return /tmp/ee.tcmd.pid }
proc tclfile {} { return /tmp/ee.tcmd.tcl }
writefile [pidfile] "[pid]\n"
trap {source [tclfile]} USR2
]]
[P and the code to send a command via SIGUSR2 to tcmd.tcl from Lua is
just this (from intro.lua):]
[PREBOXBLOGME [E'
-- From: (find-angg "DAVINCI/intro.lua")
-- See also: (find-angg "LUA/lua50init.lua" "getoutput")
tcmd = function (tclcode)
writefile("/tmp/ee.tcmd.tcl", tclcode)
local output = getoutput("kill -USR2 $(cat /tmp/ee.tcmd.pid)")
if output ~= "" then error(output) end
end
]]
[P There are several reasons for [BF not] using communication by [TT
SIGUSR2]s - it's not safe enough, that doesn't work on Windows outside
of Cygwin, "why don't you use sockets?", "why don't you use FIFOs?" -
but the code to implement communication by [TT SIGUSR2]s is trivial to
implement, so [BF that's what I will use in the first prototypes].
Later all the libraries that need to talk to one another will be
linked into a single program, so they will call one another's
functions directly.]
[RULE ----------------------------------------]
[sec 7. «peek.lua» (to ".peek.lua")
H3 peek.lua]
[P News (2007aug06): it took me 7 days to write this instead of the 4
that I expected (gaah! With so many urgent things 8-|), but a good
part of peek.lua is ready -]
[PREBOXBLOGME [E'
# (find-es "davinci" "peek.lua:doc")
# (find-angg "DAVINCI/peek.lua")
# (find-angg "DAVINCI/peek.c")
# (find-angg "DAVINCI/peek-luadecls-1.txt")
# (find-angg "DAVINCI/peek-luadecls-2.txt")
]]
[P The [HREF DAVINCI/peek.lua.html current code] is able to understand
declarations of C types in a syntax that is very close to C ([HREF
DAVINCI/peek-luadecls-2.txt.html peek-luadecls-2.txt]), but it still
has some bugs (one is clearly marked in the code; a certain
declaration of a union is parsed incorrectly - and I just learned that
gcc handles alignment in stucts in a smart way, not in the most
obvious way) and the part that will let us inspect C data structures
from Lua is not ready yet, at all -]
[RULE ----------------------------------------]
[sec 8. «gdb» (to ".gdb")
H3 On GDB, or: on little languages for discussing data structures]
[P (2007jul09)]
[# H3 GDB as a tool for exposing the internal structure of C programs]
[P I am not exactly a C programmer, but I somehow got involved with
several programs that are written in C - [_ Emacs], [_ Lua], [_
Scintilla], [_ SciTe], [_ IUP], [_ Tcl/Tk] - and whose data structures
are quite complex.]
[P Look at the figure at page 9 in [__ lua50implpaper this paper]; if
you are familiar with upvalues and closures in Lua it gives you a very
good idea of how they are implemented, but if you try to use that
figure to understand the source code you will notice that something is
missing: the names of the data structures and of the fields. For
example: the boxes in the middle are "[TT UpVal]"s, the upward arrows
are "[TT [Q ->mext]]"s, and the contents of those upvalues - if they
are strings - can be accessed with [IT (rewrite and check)] ...]
[P [IT (Write about the box representation of conses, how to extend it
with new kinds of boxes for symbols and obarrays, and to add - and
erase - addresses; discuss the [ES lua-intro#captures-data-structures
Lua-ish representation]; point to the two languages that can be used
to access the data structures in a running program with GDB: just the
commands, and commands-plus-some-gdb-output (this one would be good
for e-mails); discuss the mapping between the structures in a C
library and the Lua-visible structures than can be used from a Lua
binding for that library).]]
[_TARGETS
scriptgdb-gmane => http://article.gmane.org/gmane.emacs.eev.devel/47
scriptgdb-mlist => http://lists.gnu.org/archive/html/eev/2007-07/msg00000.html
scriptgdb-sshot => http://angg.twu.net/IMAGES/eepitch-gdb.png
scite-running-lua-shot => http://angg.twu.net/DAVINCI/scite-running-lua.png
scite-properties-file -> (find-es "scintilla" "scite-properties-file")
]
[P [IT (On scripting GDB: a message to a mailing list -
[__ scriptgdb-gmane here] and
[__ scriptgdb-mlist here].
What really matters is the
[__ scriptgdb-sshot screenshot] in it - and it
almost speaks by itself).]]
[# P [SCALEDIMAGE 25% IMAGES/eepitch-gdb.png]]
[P [SCALEDIMAGE 25% DAVINCI/eepitch-gdb.png]]
[RULE ----------------------------------------]
[H3 Scripting Scite and Scintilla using Lua]
[P [IT (Some messy notes are [__ scite-properties-file here],
and above that block in the same page; there's a screenshot - of
something trivial - [R
http://angg.twu.net/IMAGES/scite-running-lua.png here]).]]
]
]
[#
# Local Variables:
# coding: raw-text-unix
# modes: (fundamental-mode blogme-mode)
# End:
#]