Engwar

If you’ve read the architectural notes about RESTful applications and want to get your hands dirty writing one quickly, RESTful PHP Web Services by Samisa Abeysinghe is a book you should read. It starts with an introduction to what REST is and surrounding technologies. Since many web applications today expose their functions as RESTful APIs, you’ll be better prepared to play around with these after reading this book. Also a simple application that explains all the necessary functions for developing a RESTful application is discussed.

Introduction gives the reader a head start about what REST is all about and tools needed to play with RESTful web services. Then PHP libraries and function used for talking to a REST service is explained. How CURL is used to send requests with various HTTP verbs such as GET, POST, PUT, DELETE are explained with examples. After getting an XML response from a service, PHP libraries for manipulating XML and extracting parts of data you want is illustrated with samples. One could directly copy and paste these codes and try it out as they read along. Then the author goes on to explain how to use Flickr with PHP using CURL. If you’ve been using a high level library that insulate the lower level details, this example shows some of the basic API calls and how to use them.

After explaining a mashup of BBC news feed and Yahoo search API, author explains how to design RESTful services. This is carried out along with the simple but complete example of a library system. How to provide a service as well as consuming that using PHP and CURL is explained with complete code samples. Then he explains how to design the same thing using the Zend Framework.

The book concludes with a chapter dedicated to debugging REST web services. Using the TCPMon tool to capture messages and look for possible errors. The chapter ends with a set of best practices that everyone should be aware of if you do any programming with REST web services. The book also mentions WSF/PHP as an advanced framework providing many more functionality amidst of acting as a REST framework. I highly recommend this book if you’re a PHP programmer waiting to get your hands dirty designing RESTful applications. To whet your appetite here’s a sample chapter from the book about designing and implementing resource oriented clients. Enjoy!

Samisa has written a nice post about using PHP in the enterprise. A commenter there argues that he don’t use PHP because it’s a dynamically typed language. It’s somewhat true that if you’re coming from a statically typed language background you’re stormed with so many questions and wonder possible nightmares of using one. Once you start learning a language and understand how programs are suppose to be written in that language most of these problems go away.

For all these years, it has only been Java/.Net programmers who has been enjoying the luxury of service oriented architectures. These folks understand the value of building a business functionality as a service or exposing an existing functionality as a service. PHP programmers have been building their web based programs happily without having to know what services mean. People in the PHP community has tried to bring Web services into the language with varying degrees of success. These are very courages and thoughtful efforts. But if you analyze them objectively you’ll see that many of these libraries doesn’t contain that much of features in order to build/integrate an application with an existing enterprise app.

What WSF/PHP has done is lower the bar of building PHP applications that can fit with existing applications in an enterprise written in Java/.Net or whatever the language that might be. Also, it has enabled ability to integrate existing PHP applications to your other enterprise applications.

One strong reason companies have embraced developing applications in Java is that so their data will not be isolated from rest of the enterprise data. In the past having a PHP application inside your company might have been frowned upon because it doesn’t make sense to have information which is isolated from all the other applications in the company. This is why most organizations invest in an ERP for example. But now, this has become a non issue. If you’re using open standards inside your enterprise to communicate/integrate applications, using WSF/PHP, you can integrate or extend services seamlessly with rest of your enterprise services.

While developing server side components that enables providing Web services using Perl bumped into a brick wall. First the requirement is when a client request comes, need to extract the payload and pass it to WSF/C which acts as the underlying Web services engine. So WSF/C knows how to process the incoming XML payload and do what it does and return another XML payload which is then passed back to the client who send the request. And the business logic is written using Perl. Simple, I know.

We used SWIG to generate some wrapper functions (plus some custom written C code) which belongs to WSF/C and wrote a Perl module encapsulating the lower level stuff. So when you have to write a Web service in Perl all you have to do is use WSO2::WSF::Service; in your script and call a couple of functions. As the deployment scenario we choose Apache with ModPerl. That combination seemd logical and some of the heavy trafficked sites are using that.

It gets interesting when you test the whole thing. Apache is configured to run ModPerl on a folder named /perl so requests matching /perl/something will go through ModPerl. ResponseHandler is ModPerl::Registry and also GlobalRequest has been enabled to get the global request via Apache::RequestUtil->request().

Say, we have a service script named echo_service.pl which simply return back whatever it gets. Here’s the code,

#!/usr/bin/perl
 
use strict;
use WSO2::WSF;
use WSO2::WSF::WSService;
use Apache2::RequestUtil ();
 
my $payload =<<E;
<ns1:echoString xmlns:ns1="http://perl.axis2.org/samples">
  <text>plurk u can haz</text>
</ns1:echoString>
E
 
sub echoFunction {
    my $arg = $_[0];
    my $message = new WSO2::WSF::WSMessage ({'payload' => $arg});
    return $message;
}
 
my %operations = ( 'echoString' => 'echoFunction');
 
if (defined $ENV{MOD_PERL}) {
  my $r = Apache2::RequestUtil->request();
 
  my $service = new WSO2::WSF::WSService(
    {'operations' => \%operations, 'action' => 'testAction'}
  );
 
  $service->reply( $r );
}

When it goes to the reply method it’ll call up some C routines. Inside one of those C functions, after determining the function name to call by looking at the request payload, we need to call it. From a C function. How can you do that? By embedding the Perl interpreter. AFAIK, you cannot call a Perl function from C without embedding the interpreter (please correct me if I’m wrong). In WSF/Ruby server side there was no such issue because the entire Ruby module (WSService class and methods) is written in C.

Let me show you the code that embeds the interpreter and calls the function after looking at the XML payload in the request,

static axiom_node_t *
wsf_xml_msg_recv_invoke_other(axis2_msg_recv_t* msg_recv,
        const axutil_env_t* env,
        wsf_svc_info_t* svc_info,
        axis2_msg_ctx_t* in_msg_ctx,
        axis2_msg_ctx_t* out_msg_ctx,
        axis2_char_t* function_name,
        axis2_char_t* class_name)
{
    AXIS2_PARAM_CHECK(env->error, svc_info, NULL);
    AXIS2_PARAM_CHECK(env->error, in_msg_ctx, NULL);
    AXIS2_PARAM_CHECK(env->error, out_msg_ctx, NULL);
 
    axiom_node_t *node = NULL;
    axiom_node_t *om_node = NULL;
    axiom_soap_envelope_t *envelope = NULL;
    axiom_soap_body_t *body = NULL;
    axis2_char_t *retstr = NULL;
    axiom_node_t *soap_body_node = NULL;
 
    /* extracting payload from the soap message */
    envelope = axis2_msg_ctx_get_soap_envelope(in_msg_ctx, env);
    if (!envelope)
    {
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, "[wsf-perl-service] soap envelope not found");
        return NULL;
    }
 
    body = axiom_soap_envelope_get_body(envelope, env);
    if (!body)
    {
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, "[wsf-perl-service] soap body not found");
        return NULL;
    }
 
    soap_body_node = axiom_soap_body_get_base_node(body, env);
    if (!soap_body_node)
    {
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, "[wsf-perl-service] soap body base node not found");
        return NULL;
    }
 
    om_node = axiom_node_get_first_child(soap_body_node, env);
    if (!om_node)
    {
        return NULL;
    }
 
    axis2_char_t *embedding[] = {"-M'WSO2::WSF::C; WSO2::WSF::Service;'", ""};
    if (!svc_info->script_filename)
    {
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI,
                "perl function invocation failed, script_file name not found for "
                "service %s", svc_info->svc_name);
        return NULL;
    }
    /* passing script real path into perl interpreter. */
    embedding[1] = svc_info->script_filename;
 
    my_perl = perl_alloc();
    PL_perl_destruct_level = 0;
/*    PL_use_safe_putenv = 1; */
    PERL_SET_CONTEXT(my_perl);
    perl_construct(my_perl);
 
    PL_origalen = 1;
    PERL_SET_CONTEXT(my_perl);
    if (perl_parse(my_perl, xs_init, 2, embedding, NULL))
    {
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, "perl_parse method failed");
        return NULL;
    }
    PL_exit_flags |= PERL_EXIT_DESTRUCT_END;
    /* perl_run(my_perl); */
 
    if (SvTRUE(ERRSV))
    {
        AXIS2_LOG_DEBUG(env->log, AXIS2_LOG_SI, "invoke perl function failed");
    }
 
    retstr = invoke_perl_function(env, om_node, function_name, NULL);
 
    if (retstr)
    {
        node = wsf_util_deserialize_buffer(env, retstr);
    }
 
    PERL_SET_CONTEXT(my_perl);
    perl_destruct(my_perl);
    PERL_SET_CONTEXT(my_perl);
    perl_free(my_perl);
    PERL_SYS_TERM();
    AXIS2_LOG_DEBUG(env->log, AXIS2_LOG_SI, axiom_node_to_string(node, env));
    return node;
}
 
static axis2_char_t *
invoke_perl_function(const axutil_env_t *env, axiom_node_t *om_node,
        axis2_char_t *operation, axis2_char_t *class_name)
{
    int count = 0;
    axis2_char_t *inmsg = NULL;
    SV **wsmsg_str = NULL; /* hold the value for the key 'payload' in WSMessage */
    SV *wsmsg_ref = NULL; /* reference to a WSMessage object */
    HV *wsmsg = NULL; /* WSMessage object */
    char *tmp_str = NULL;
    axis2_char_t *res_payload_str = NULL;
 
    if (!operation)
    {
        AXIS2_LOG_DEBUG(env->log, AXIS2_LOG_SI,
                "invoking perl function failed, operation not available");
        return NULL;
    }
 
    if (om_node)
    {
        inmsg = axiom_node_to_string(om_node, env);
    }
 
    /* declare and init a local copy of the Perl stack pointer */
    dSP;
 
    /* mortal SVs for the stack */
    ENTER;
    SAVETMPS;
 
    /* "record" the current stack pointer */
    PUSHMARK(SP);
 
    /* push parameters to the stack */
    XPUSHs(sv_2mortal(newSVpv(inmsg, 0)));
 
    /* make the global copy of the stack pointer same as the local copy */
    PUTBACK;
 
    /* call the Perl function, expecting a scalar to be returned */
    count = call_pv(operation, G_SCALAR);
 
    /* refreshing the local copy of the stack pointer, call_pv might have reallocated it */
    SPAGAIN;
 
    if (count != 1)
    {
        croak("perl function invocation failed") ;
        AXIS2_LOG_ERROR(env->log, AXIS2_LOG_SI, "perl function %s invocation failed", operation);
    }
 
    /* get the scalar reference from the stack */
    wsmsg_ref = (SV *) POPs;
 
    /* getting the object from the reference */
    wsmsg = SvRV(wsmsg_ref);
 
    /* fetching the value of the member variable 'payload' from WSMessage */
    wsmsg_str = hv_fetch(wsmsg, "payload", 7, FALSE);
 
    /* perl scalar to c string */
    tmp_str = SvPVutf8_nolen(*wsmsg_str);
    res_payload_str = (axis2_char_t *) savepvn(tmp_str, strlen(tmp_str));
 
    PUTBACK;
 
    /* cleaning up mortal SVs */
    FREETMPS;
    LEAVE;
 
    return res_payload_str;
}

Full source is here (note that the actual source might be different from what you see here since it’s being modified at the time of this writing). Also my_perl is declared as a static variable. The problem is when my_perl gets freed it looks like ModPerl is also affected. A valgrind report shows that it tries to read from a pointer after perl_free() gets called. As the following valgrind exerpt shows,

==16101== Invalid read of size 4
==16101==    at 0x62AD118: _wrap_wsf_worker_process_request (wsf_wrap.c:4167)
==16101==    by 0x5476D68: Perl_pp_entersub (pp_hot.c:2847)
==16101==    by 0x5438092: Perl_runops_debug (dump.c:1931)
==16101==    by 0x546FF97: Perl_call_sv (perl.c:2646)
==16101==    by 0x4DC2AF3: modperl_callback (modperl_callback.c:101)
==16101==    by 0x4DC388B: modperl_callback_run_handlers (modperl_callback.c:262)
==16101==    by 0x4DC4289: modperl_callback_per_dir (modperl_callback.c:369)
==16101==    by 0x4DBB53E: modperl_response_handler_run (mod_perl.c:1004)
==16101==    by 0x4DBB704: modperl_response_handler_cgi (mod_perl.c:1099)
==16101==    by 0x25A6C: ap_run_handler (config.c:158)
==16101==    by 0x295BE: ap_invoke_handler (config.c:372)
==16101==    by 0x35D80: ap_process_request (http_request.c:258)
==16101==  Address 0x649051c is 12 bytes inside a block of size 1,692 free'd
==16101==    at 0x480590A: free (vg_replace_malloc.c:323)
==16101==    by 0x546AED4: perl_free (perl.c:1394)
==16101==    by 0x62BD599: wsf_xml_msg_recv_invoke_business_logic_sync (wsf_xml_msg_recv.c:452)
==16101==    by 0x613D503: axis2_msg_recv_invoke_business_logic (msg_recv.c:392)
==16101==    by 0x613DB34: axis2_msg_recv_receive_impl (msg_recv.c:319)
==16101==    by 0x613D583: axis2_msg_recv_receive (msg_recv.c:431)
==16101==    by 0x61327D4: axis2_engine_receive (engine.c:315)
==16101==    by 0x6167177: axis2_http_transport_utils_process_http_post_request (http_transport_utils.c:595)
==16101==    by 0x62BE9B3: wsf_worker_process_request (wsf_worker.c:301)
==16101==    by 0x62AD0F7: _wrap_wsf_worker_process_request (wsf_wrap.c:4166)
==16101==    by 0x5476D68: Perl_pp_entersub (pp_hot.c:2847)
==16101==    by 0x5438092: Perl_runops_debug (dump.c:1931)

Still haven’t been able to pin down what exactly happens when perl_free() gets called