ayrton: an update
«So, by simply copying that one and ast.py
, modifying the latter so it
uses my modified syntax checker, and modifying the setup.py
file, I should be
done». Famous last words.
I was completely wrong. The _ast
module imported by ast.py
is not implemented
in Python/ast.c
but in Python/Python-ast.c
. Understanding this, I reread this
post
I linked to last time and finally1 understood that changing the syntax would
not be so easy. Another eye-opener was the stack trace I got when ast.c
's
ast_for_call()
(the function I'm planning to change) is called when a function
call is found in the source. Here, have a look:
#0 ast_for_call (c=0x7fffffffa300, n=0x7ffff700d918, func=0x9ccf40) at Python/ast.c:2470 #1 0x00000000005bfe73 in ast_for_trailer (c=0x7fffffffa300, n=0x7ffff700db98, left_expr=0x9ccf40) at Python/ast.c:2105 #2 0x00000000005c03c0 in ast_for_power (c=0x7fffffffa300, n=0x7ffff6ca4238) at Python/ast.c:2217 #3 0x00000000005c0b01 in ast_for_expr (c=0x7fffffffa300, n=0x7ffff6ca4238) at Python/ast.c:2406 #4 0x00000000005c1108 in ast_for_testlist (c=0x7fffffffa300, n=0x7ffff6ca4030) at Python/ast.c:2562 #5 0x00000000005c118e in ast_for_expr_stmt (c=0x7fffffffa300, n=0x7ffff6facfa8) at Python/ast.c:2584 #6 0x00000000005c3e84 in ast_for_stmt (c=0x7fffffffa300, n=0x7ffff6facfa8) at Python/ast.c:3586 #7 0x00000000005bc94c in PyAST_FromNodeObject (n=0x7ffff6face18, flags=0x7fffffffa4e0, filename=0x7ffff6d56370, arena=0x8c8780) at Python/ast.c:709 #8 0x000000000050c9e8 in PyParser_ASTFromStringObject (s=0x7ffff6ca37a0 "foo (a=3, b)", filename=0x7ffff6d56370, start=257, flags=0x7fffffffa4e0, arena=0x8c8780) at Python/pythonrun.c:2160 #9 0x000000000050c6be in Py_CompileStringObject (str=0x7ffff6ca37a0 "foo (a=3, b)", filename=0x7ffff6d56370, start=257, flags=0x7fffffffa4e0, optimize=-1) at Python/pythonrun.c:2068 #10 0x00000000004c6e54 in builtin_compile (self=0x7ffff70f9c28, args=0x7ffff6ca04f8, kwds=0x0) at Python/bltinmodule.c:671
The compile()
builtin (#10) passes through a couple of obscure functions till
it reaches the AST build/syntax check code (all the ast_for_*()
functions), and
we quickly reach it from ast.parse()
:
def parse(source, filename='<unknown>', mode='exec'): return compile(source, filename, mode, PyCF_ONLY_AST)
In a desperate move, I though of hacking a modified version of just that function
and do some LD_PRELOAD
jedi moves, but the Python interpreter binary (/usr/bin/python3
)
is statically linked against libpython3
. Future versions2 of the Python
interpreter main()
function3will be really simple:
/* Minimal main program -- everything is loaded from the library */ #include "Python.h" #ifdef __FreeBSD__ #include <floatingpoint.h> #endif int main(int argc, char **argv) { /* 754 requires that FP exceptions run in "no stop" mode by default, * and until C vendors implement C99's ways to control FP exceptions, * Python requires non-stop mode. Alas, some platforms enable FP * exceptions by default. Here we disable them. */ #ifdef __FreeBSD__ fp_except_t m; m = fpgetmask(); fpsetmask(m & ~FP_X_OFL); #endif return Py_Main(argc, argv); }
So it's not so crazy to think of creating a new interpreter that uses my version
of ast_for_call()
, but I haven't found the linking command that makes that
possible. So this part of the project is in the freezer.
Meanwhile, I spent a couple of hours with the code and managed to implement piping and redirection, even with some tests! You can have an idea of how it works reading the README4. Also, expect a release soon.
As for replacing sh
, no efforts (besides burning some glucose and firing some
neurons) have been done.
-
Also famous last words. ↩
-
Python-3.3 has a more complex version which has to handle making two copies of
argv
while dealing with the system's encoding. ↩ -
It's weird, the source code of the executable resides in
Modules/python.c
. Go figure. ↩ -
Some day I'll convert that file to reST, make it the tutorial, and have it with the rest of the documentation. ↩