HTTP POST/PUT request bodies

This page describes PGL knobs that control generation of HTTP request bodies. These knobs are available starting with Polygraph version 3.3.

Table of Contents

1. For the impatient
2. Introduction
3. Request body content configuration
4. Paused requests
    4.1 Client side configuration
    4.2 Server side configuration
5. Old workloads

1. For the impatient

// Content for POST and PUT request bodies.
Content SimpleContent = {
    size = exp(13KB); // request body size, exponential distribution
    mime = { // optional MIME parameters for the body
        type = "text/html"; // request body content type
        prefixes = [ "page" ]; // used to enable Content-Disposition
        extensions = [ ".html" ]; // filename value of "page*.html"
    };
    checksum = 50%; // half of request bodies come with a checksum
};

// Robot using the above Content to make POST and PUT bodies
Robot R = {
    req_methods = ["GET", "POST": 15%, "PUT": 3% ]; // 18% with bodies
    req_body_pause_prob = 50%; // add "Expect: 100-continue" header
    put_contents = [ SimpleContent ]; // what content to use for PUT
    post_contents = put_contents; // use same POST body content
    ...
};

// 70% of paused requests will be accepted with a "100 Continue" reply.
// Other paused requests will be rejected with 417 Expectation Failed.
Server S = {
    req_body_allowed = 70%; // affects "Expect: 100-continue" requests
    ...
};

2. Introduction

Most HTTP requests are GET requests without bodies. However, simulating requests with bodies is important to properly stress the proxy code and to test various hooks working with such requests. Most HTTP requests with bodies use POST or PUT request method. Both methods are supported by Polygraph.

Polygraph offers great control and flexibility in how request message body is generated. It is configured in a similar way to response content generation.

In addition to simulating request bodies, Polygraph supports "100 Continue" control messages on both client and server side. The implementation follows Section 8.2.3 of RFC 2616.

3. Request body content configuration

To generate requests with bodies, POST or PUT request method should be configured using Robot's req_method field. The actual content is controlled with post_contents and put_contents arrays. These selectors contain Content PGL objects that are used to generate message bodies.

For simple, randomly generated bodies, the following sketch may be sufficient:

// Random body with a length from 50KB to 100KB.
Content cntSimple = {
    size = unif(50KB, 100KB);
};
Robot R = {
    req_methods = [ "POST" ]; // usually more than just POSTs
    post_contents = [ cntSimple ];
    ...
};

The prefix and extensions specified in Content's mime field can be used to trigger generation of a Content-Disposition request header using the following format: 

Content-Disposition: attachment; filename="prefix*.extension"
If no mime is set, or neither prefix nor extensions are configured, the Content-Disposition request header is not sent.

Request body generator supports all applicable features of response content generation. For example, if you want content databases can be used to generate request bodies with specific files or even injected text:

Content cntInfoLeak = {
    mime = { type = "text/html"; extensions = [ ".html" ]; };
    content_db = "sensitive.cdb"; // import collected content
};
Robot R = {
    req_methods = ["POST", "GET": 90% ];
    put_contents = [ cntInfoLeak: 1%, cntSimple ]; // rare leaks
};

Content cachable and obj_life_cycle knobs are not applicable to request body generation.

4. Paused requests

Polygraph robots can be configured to send "Expect: 100-continue" header for requests with bodies (see RFC 2616 section 8.2.3). Requests with that header are sometimes called "paused requests" because the client sends the request header and then pauses, expecting the server confirmation that the body should follow the headers.

Polygraph servers can either "accept" or "reject" a request body. If the request body is accepted, an HTTP 100 "Continue" control message is sent to the client. The server then reads request body and writes the final response as usual. If the server rejects the request body, it sends an HTTP 417 "Expectation Failed" response.

Robots and servers are configured independently. This allows simulating broken HTTP agents as well as testing proxies that add "Expect: 100-continue" headers.

4.1 Client side configuration

Client side configuration is done via the Robot PGL type. There are two mutually exclusive parameters that control when an "Expect: 100-continue" header is added: The req_body_pause_prob parameter specifies probability that the "Expect: 100-continue" header is added. The req_body_pause_start sets the minimum request body size for which the header is generated (smaller requests will not have the header).

If you want half of requests with body to have an "Expect: 100-continue" header, consider the following configuration:

Robot R = {
    req_methods = [ "POST", "GET" ];
    post_contents = [ SimpleContent ];

    req_body_pause_prob = 50%;
};

If you want to add "Expect: 100-continue" header to all requests with bodies larger than 50KB, the following should work:

Robot R = {
    req_methods = ["PUT", "GET" ];
    put_contents = [SimpleContent ];

    req_body_pause_start = 50KB;
};

When neither req_body_pause_prob nor req_body_pause_start parameter is configured, Polygraph robots do not add the "Expect: 100-continue" header. This is the default.

4.2 Server side configuration

Server side configuration is done via the Server PGL type: The req_body_allowed parameter specifies the probability that a paused request (i.e., the request with an "Expect: 100-continue" header) is accepted.

To accept half of requests with the "Expect: 100-continue" header, use the following configuration sketch:

Server S = {
    req_body_allowed = 50%;
    ...
};

When req_body_allowed parameter is not set, Polygraph server accepts all paused requests. This is the default.

5. Old workloads

Polygraph versions prior to 3.3 had hard-coded request content generation. Request body length was determined using a uniform distribution and varied between 0 and 8KB. The content was randomly generated.

To mimic old behavior, try the following approach:

Content cntOld = {
    size = unif(0KB, 8KB);
};
Robot R = {
    req_methods = [ "POST", "PUT", ... ];
    post_contents = [ cntOld ];
    put_contents = post_contents;
    ...
};

Standard Polygraph workloads developed before request bodies became configurable were adjusted using the above approach.