Perl Chroot'd Environment

Z

zoom

Well-Known Member
#1
#1
I downloaded and compiled perl with the install path pointing to my chroot'd webroot /chroot/usr/local . I then setup a new external application that uses the lsperl fastcgi module and created a script handler for it. When I attempt to run a very simple hello world perl script "test.pl" I get "503 Service Unavailable"??? I checked the perl binary located in the chroot to see what shared libraries it was using and verified that those libraries are located in the chroot as well.

Code: 
    libnsl.so.1 => /lib/libnsl.so.1 (0x40026000)
        libdl.so.2 => /lib/libdl.so.2 (0x4003b000)
        libm.so.6 => /lib/tls/libm.so.6 (0x4003f000)
        libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
        libcrypt.so.1 => /lib/libcrypt.so.1 (0x40062000)
        libutil.so.1 => /lib/libutil.so.1 (0x4008f000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
All the perl binaries, libs etc are located within the chroot. I'm I missing something?
 
M

mistwang

LiteSpeed Staff
#2
#2
In addition to the dynamic libraries showed in ldd result, Perl also dynamically load libraries and modules at runtime.

You must make sure a command like following works in your chroot jail

chroot /chroot perl <your script>

If you cannot tell which library is missing from the result, you have to analyze results of "strace perl <your script>" running inside the jail and in mornal environment.

You can try "chroot.sh perl", it will try to setup a perl chroot environment by copying Perl in the Linux distribution.
 
Z

zoom

Well-Known Member
#3
#3
Thanks for the suggestions mistwang. The script did run successfully using the chroot command. I double checked all the shared libs reported by strace as well. I did notice something strange in the stderr.log

lscgid: execve(): No such file or directory
Click to expand...
I checked the lsperl fastcgi module and it seems it was pointing to /usr/bin/perl, however the perl binary is located in /usr/local/bin. I changed the shebang to use #!/usr/local/bin/perl then loaded the FCGI.pm module using cpan. Now I'm getting the following message:
Code: 
Can't locate FCGI.pm in @INC (@INC contains: /chroot/usr/local/lib/perl5/5.6.2/i686-linux /chroot/usr/local/lib/perl5/5.6.2 /chroot/usr/local/lib/perl5/site_perl/5.6.2/i686-linux /chroot/usr/local/lib/perl5/site_perl/5.6.2 /chroot/usr/local/lib/perl5/site_perl .) at /opt/lsws/fcgi-bin/lsperld.fpl line 3.
Why can't it locate FCGI.pm??? It's under the chroot??
 
Z

zoom

Well-Known Member
#5
#5
mistwang,
Still having some problems getting Perl to work in a chroot'd environment using the Pro version. I've copied /usr/lib/perl5 directory into my chroot directory /chroot/usr/lib/perl along with all the any binaries etc. running the command "chroot /chroot perl -V" reports back the following

/usr/lib/perl5/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/5.8.0
/usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/site_perl/5.8.0
/usr/lib/perl5/site_perl
/usr/lib/perl5/vendor_perl/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/vendor_perl/5.8.0
/usr/lib/perl5/vendor_perl
/usr/lib/perl5/5.8.0/i386-linux-thread-multi
/usr/lib/perl5/5.8.0

Which is good since that's where all my libs would be under the chroot. The ldd command reports back the following shared libs used by perl. All the shared libs exist under the chroot directory

libperl.so => /usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE/libperl.so (0x40017000)
libnsl.so.1 => /lib/libnsl.so.1 (0x4014e000)
libdl.so.2 => /lib/libdl.so.2 (0x40163000)
libm.so.6 => /lib/tls/libm.so.6 (0x40168000)
libpthread.so.0 => /lib/tls/libpthread.so.0 (0x4018a000)
libc.so.6 => /lib/tls/libc.so.6 (0x42000000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x40198000)
libutil.so.1 => /lib/libutil.so.1 (0x401c5000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x4000000

Running the script under the chroot using the command "chroot /chroot perl /www/html/test.pl" replies with

Content-type: text/html
hello, world

As you can see the script is quite simple and runs fine using the chroot command. However, it fails to run under LiteSpeed. LiteSpeed reports back "503 Service Unavailable"

The logs for the web are reporting the following messages when I set them to DEBUG level

www.mydomain.net] processContextPath() return 0
www.mydomain.net] run lsapi processor.
www.mydomain.net:lsapi] [ExtConn] reconnect()
www.mydomain.net:lsapi] ExtConn::eek:nWrite()
www.mydomain.net:lsapi] request header is done
www.mydomain.net:lsapi] ExtConn::continueRead()
www.mydomain.net:lsapi] Request body done!
www.mydomain.net:lsapi] ExtConn::suspendWrite()
www.mydomain.net:lsapi] processNewReq() return 0.
www.mydomain.net:lsapi] readToHeaderBuf() return 0.
www.mydomain.net:lsapi] ExtConn::eek:nRead()
www.mydomain.net:lsapi] LsapiConn::doRead()
www.mydomain.net:lsapi] process packet header -1 bytes
www.mydomain.net:lsapi] connection to [uds://tmp/lshttpd/lsperl.sock] on request #1, error: Connection reset by peer!

Any ideas why it's failing with lsws???
 
M

mistwang

LiteSpeed Staff
#6
#6
I think there still are missing libraries/files that used by perl at runtime, you can try run the lsperd inside jail, it may require a fcgi start, like the "cgi-fcgi" comes with the fastcgi-2.4 package. Please refer its man page.

you can try "chroot /chroot strace cgi-fcgi -start <socketaddr> <path_to_lsperld.fpl>", and you can do the same outside of the jail and compare the output.

chroot.sh trys to copy those required files into the jail, it works on Redhat 9, but file locations is different in different linux distribution, you may read the chroot.sh and make sure all files has been copied over.
 
Z

zoom

Well-Known Member
#7
#7
mistwang,
I don't believe the problem is with the perl environment setup in the chroot. The reason is that if I execute the script as a CGI vs lsws it works fine. I seem to be only getting the failure when attempting to use the lsperld.fpl script to run it as fastcgi.

After running lsperld.fpl as you suggested and comparing the output from each I could only find one small difference. Running in the chroot the strace revealed "--- SIGCHLD (Child exited) @ 0 (0) ---". This was the only difference from the entire trace. I compared line by line and verified each lib reference that was used.

lsperld.fpl run outside of chroot
Code: 
listen(3, 5)                            = 0
access("/chroot/opt/lsws/fcgi-bin/lsperld.fpl", X_OK) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|0x11, <ignored>, <ignored>, 0x40044748) = 20335
shutdown(3, 1 /* send */)               = 0
select(4, [3], NULL, NULL, {2, 0}
)      = 0 (Timeout)
close(3)                                = 0
exit_group(0)                           = ?
lsperld.fpl run inside the chroot environment
Code: 
listen(3, 5)                            = 0
access("/opt/lsws/fcgi-bin/lsperld.fpl", X_OK) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|0x11, <ignored>, <ignored>, 0x40044748) = 20315
--- SIGCHLD (Child exited) @ 0 (0) ---
shutdown(3, 1 /* send */)               = 0
select(4, [3], NULL, NULL, {2, 0}
)      = 0 (Timeout)
close(3)                                = 0
exit_group(0)                           =
I'm not sure what I should be looking for now, however I think if I can find another fastcgi script I'll validate it against my environment setup. If it works then I know the problem is with lsperld.fpl somewhere..
 
Z

zoom

Well-Known Member
#9
#9
mistwang,
I'm doing my testing on a Redhat 9 system. I've tried copying the perl libs from the default RH install, same problem.

Currently I'm testing with a version I compiled and installed (Perl 5.8.7). For the build, I compiled and installed Perl under /usr/local/. Once complete I copied the libs from /usr/local/perl into the chroot /chroot/usr/local/perl. I also moved all the binaries into there respective location as well.

I tested perl under the chroot using "chroot /chroot perl -V" and by running a couple simple scripts under the chroot as well. Everything seems fine and working.

I then modified the lsperld.fpl shebang to use #!/usr/local/bin/perl instead of #!/usr/bin/perl. However I still get the 503 error.

I've been able to run Perl scripts under CGI in the chroot'd environment without any problems via LSWS but creating a context definition. I even tried running a very simple Perl script that uses FCGI and it runs fine under CGI (see example) using LSWS.

Code: 
#!/usr/local/bin/perl -w 

use FCGI;
while(FCGI::accept() >= 0) {
    print("Content-type: text/html\r\n\r\n",
          "<title>FastCGI Hello! (Perl)</title>\n",
          "<h1>FastCGI Hello! (Perl)</h1>\n");
}
I might try a clean install from scratch this weekend if I can't get it working on my test box. I'm sure it's just a missing library somewhere as you suggested. The problem is finding it. I didn't think chrooting a webserver was this much work, however in the end it'll be worth it from a security standpoint. Thanks again for your continued assistance.
 
M

mistwang

LiteSpeed Staff
#10
#10
I think you are pretty close to figure out the missing libraries.

When you straced the lsperld.fpl, you actually traced the cgi-fcgi or whatever the fcgi spawner you used, not the lsperld.fpl itself. You should either let strace to trace children process by adding an command line option, or do something like "cgi-fcgi ... strace ./lsperld.fpl".

Once you master how to use strace properly, you can figure out this kind of problem easily, no matter it is perl, python, or ruby, etc.

Please let me know what libraries/files are missing after you figured it out. :)
 
Z

zoom

Well-Known Member
#11
#11
mistwang,
Thanks for the assistance. Instead of spending countless hours trying to figure out which library it was causing the issue I just reinstalled RH9, LSWS and Perl. Setup the chroot environment for LSWS and copied all the Perl stuff into the chroot enviroment. It worked perfectly..

I just hope that it goes this smooth for me on the production server.. That's one machine I can't reinstall from scratch. :lol:
 
You must log in or register to reply here.
Top