how to read linux kernel source with pycparser

[meInvent bbird]

is it possible to git pull a part of directory such as sched
and compile this subdirectory and pycparser it?

i got error when run gcc -E

git submodule add hello at github.com:hello/repo.git kernel/sched
cd kernel/sched
git pull

martin at ubuntu:~/Downloads/kernel/sched$ gcc -E -std=c99 *.c
# 1 "auto_group.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "auto_group.c"
# 1 "sched.h" 1
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "sched.h" 2
In file included from auto_group.c:1:0:
sched.h:2:32: fatal error: linux/sched/sysctl.h: No such file or directory
compilation terminated.
# 1 "clock.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "clock.c"
clock.c:55:28: fatal error: linux/spinlock.h: No such file or directory
compilation terminated.
# 1 "completion.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "completion.c"
# 14 "completion.c"
# 1 "/usr/include/linux/sched.h" 1 3 4
# 15 "completion.c" 2
completion.c:15:30: fatal error: linux/completion.h: No such file or directory
compilation terminated.
# 1 "core.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "core.c"
core.c:29:25: fatal error: linux/kasan.h: No such file or directory
compilation terminated.
# 1 "cpuacct.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "cpuacct.c"
cpuacct.c:1:26: fatal error: linux/cgroup.h: No such file or directory
compilation terminated.
# 1 "cpudeadline.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "cpudeadline.c"
cpudeadline.c:14:23: fatal error: linux/gfp.h: No such file or directory
compilation terminated.
# 1 "cpufreq.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "cpufreq.c"
# 12 "cpufreq.c"
# 1 "sched.h" 1
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "sched.h" 2
In file included from cpufreq.c:12:0:
sched.h:2:32: fatal error: linux/sched/sysctl.h: No such file or directory
compilation terminated.
# 1 "cpufreq_schedutil.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "cpufreq_schedutil.c"
cpufreq_schedutil.c:14:27: fatal error: linux/cpufreq.h: No such file or directory
compilation terminated.
# 1 "cpupri.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "cpupri.c"
cpupri.c:30:23: fatal error: linux/gfp.h: No such file or directory
compilation terminated.
# 1 "cputime.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "cputime.c"
cputime.c:1:26: fatal error: linux/export.h: No such file or directory
compilation terminated.
# 1 "deadline.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "deadline.c"
# 17 "deadline.c"
# 1 "sched.h" 1
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "sched.h" 2
In file included from deadline.c:17:0:
sched.h:2:32: fatal error: linux/sched/sysctl.h: No such file or directory
compilation terminated.
# 1 "debug.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "debug.c"
debug.c:13:27: fatal error: linux/proc_fs.h: No such file or directory
compilation terminated.
# 1 "fair.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "fair.c"
# 23 "fair.c"
# 1 "/usr/include/linux/sched.h" 1 3 4
# 24 "fair.c" 2
fair.c:24:30: fatal error: linux/latencytop.h: No such file or directory
compilation terminated.
# 1 "idle.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "idle.c"



# 1 "/usr/include/linux/sched.h" 1 3 4
# 5 "idle.c" 2
idle.c:5:23: fatal error: linux/cpu.h: No such file or directory
compilation terminated.
# 1 "idle_task.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "idle_task.c"
# 1 "sched.h" 1
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "sched.h" 2
In file included from idle_task.c:1:0:
sched.h:2:32: fatal error: linux/sched/sysctl.h: No such file or directory
compilation terminated.
# 1 "loadavg.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "loadavg.c"
loadavg.c:9:26: fatal error: linux/export.h: No such file or directory
compilation terminated.
# 1 "rt.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "rt.c"





# 1 "sched.h" 1
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "sched.h" 2
In file included from rt.c:6:0:
sched.h:2:32: fatal error: linux/sched/sysctl.h: No such file or directory
compilation terminated.
# 1 "stats.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "stats.c"
stats.c:1:24: fatal error: linux/slab.h: No such file or directory
compilation terminated.
# 1 "stop_task.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "stop_task.c"
# 1 "sched.h" 1
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "sched.h" 2
In file included from stop_task.c:1:0:
sched.h:2:32: fatal error: linux/sched/sysctl.h: No such file or directory
compilation terminated.
# 1 "swait.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "swait.c"
# 1 "/usr/include/linux/sched.h" 1 3 4
# 2 "swait.c" 2
swait.c:2:25: fatal error: linux/swait.h: No such file or directory
compilation terminated.
# 1 "wait.c"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "wait.c"
wait.c:6:24: fatal error: linux/init.h: No such file or directory
compilation terminated.



On Wednesday, October 5, 2016 at 7:48:17 AM UTC+8, Michael Torrie wrote:
> On 10/04/2016 03:36 AM, meInvent bbird wrote:
> > i expect to use pycparser to read linux kernel source
> > and get a AST tree, 
> > 
> > but there are so many directory, 
> > 
> > how to read linux kernel source with pycparser?
> > 
> > how to customize pycparser to search what we want such as bug or fix 
> > to make a linux patch for linux kernel source with python?
> 
> C projects with many .c files aren't meant to be compiled into one unit
> (AST) usually.  The kernel is designed to be compiled into many
> discrete, compiled object files which are then linked together after
> compilation.  Each compilation unit would come from its own AST tree.
> Furthermore, most C files can't be parsed by a compiler or parser at all
> until the preprocessor has run over it first to handle the many
> #define's, #if's, etc.  Fortunately you can run the preprocessor by
> itself and output the bare C code.  On gcc I think if you pass -E it
> will output the processed code.  Or use the cpp binary, which is
> normally invoked by the compiler.
> 
> Another thing that will make this very difficult (and is related to the
> preprocessor stuff) is that the Linux kernel's compilation can take many
> different paths depending on how you configure the kernel. Some parts
> may be skipped over entirely, other parts depend on which platform you
> are configuring it for.
> 
> You could probably do this, and get lots of ASTs you can look at, but
> you'll have to do some heavy-duty scripting and modification of
> Makefiles to get it to happen in any sort of automatic way.  You make be
> able to modify the Makefiles to have GCC itself dump the parse trees as
> it creates them. I know GCC can do this. This is really the only
> practical way I can see.