Varnish

As well as being an extremely effective caching solution in it's own right, Evolved Caching also integrates directly with Varnish by way of it's caching key cookie. This cookie is generated for every page as you navigate through your site and it contains a unique hash which describes many things about the way that page is displayed, such as the current store, the currency, any category or layered navigation filtering applied, whether the page is secure or insecure, the browser user agent, the tax rate according to the customers address, and of course the URL.

Varnish is designed to cache by URL, but this results in an incomplete solution in the case of Magento as quite a few aspects which govern the way a page is displayed are stored in a users session and won't necessarily appear in the URL at all (category page filtering is one example of this). This means that Varnish can end up serving the wrong content to the user as it doesn't account for, amongst other things, session data.

The caching key cookie however is built using logic which considers all of the different ways in which this kind of data can effect a page, it's the same logic in fact Evolved Caching uses to store it's own cached content. So with Varnish using the unique hash contained in the cookie, you have an effective, complete Varnish caching solution driven by the complex logic Evolved Caching already has built in.

As well as the caching key cookie meaning that you don't need to take on the significant task of providing Varnish with enough data to cache properly, it also tells Varnish which pages it should and shouldn't cache according to the excluded pages you configure in admin.

The end result of all this is that your Varnish configuration file only needs to be relatively few lines as all Varnish needs to do is cache and serve content according to the content of the cookie.

Although you can find VCL files below, we won't cover the initial installation and setup of Varnish here. Installation is straightforward and covered for a number of different Linux distributions in the Varnish documentation. Initial setup is again simple and for many configurations, nothing more than is outlined under the installation guide will be required.

With Varnish installed and setup, you then need to edit your Varnish configuration file, which unless you have changed it during setup, can be found here on your server.

/etc/varnish/default.vcl

Again the Varnish documentation has plenty of information about building a VCL in case you want to modify the configuration below, but you should only do this if you have a specific requirement to do so. The provided VCL is sufficient to provide complete caching without requiring any changes.

So all you need to do is open default.vcl at the location above (or the one you specified if you changed it), delete it's contents and paste in the relevant VCL from below.

Varnish VCL configuration files

Edit the backend default section at the top to point at the reconfigured web server (as set during the initial Varnish setup), and also edit the three instances of 120s in the vcl_fetch/vcl_backend_response section at the bottom to however long you want HTML, CSS/JS and images to remain in the Varnish cache. This is set at the default of 120 seconds, but you will very likely want to increase it. How much you increase it by really depends on your caching strategy and the size of cache you want to make available to Varnish, but the longer you hold items in cache, the larger your cache is going to be. A side note is that you can use values other than seconds so the following would for instance tell Varnish to hold items in cache for a week.

set beresp.ttl = 1w;

With configuration complete enable the caching key cookie in the Evolved Caching admin settings, clear the Magento cache (including the full page cache), and Varnish will begin caching and serving full page HTML along side Evolved Caching. If you are unsure whether or not Varnish is serving cached content you can use a tool like Firebug to view the response headers from the server. As you navigate between pages you will see an X-Varnish header in the responses and this will contain numbers. One number string indicates a cache miss meaning Varnish did not serve cached content for that request, while two number strings indicates a cache hit meaning Varnish did serve cached content for that request.

Varnish 4

vcl 4.0;

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    if (req.method == "BAN_HTML") {
        ban("obj.http.Evolved-Type == html");
        return (synth(200, "ban_html"));
    } else if (req.method == "BAN_IMAGES") {
        ban("obj.http.Evolved-Type == image");
        return (synth(200, "ban_images"));
    } else if (req.method == "BAN_CSSJS") {
        ban("obj.http.Evolved-Type == cssjs");
        return (synth(200, "ban_cssjs"));
    } else if (req.method == "PURGE_SINGLE") {
        return (purge);
    } else {
        if (req.url ~ "/form_key/" && req.http.Cookie ~ "evolved_formkey") {
            set req.url = regsub(req.url, "/form_key/[^/]*/", "/form_key/" + regsub(req.http.Cookie, ".*evolved_formkey=([^;]+).*", "") + "/");
        }
        if (req.http.Cookie !~ "evolved_key") {
            return (pass);
        }
    }
    return (hash);
}

sub vcl_purge {
    return (synth(200, "purge_single"));
}

sub vcl_hash {
    if (req.url ~ "\.(png|gif|jpg|jpeg|swf|css|js)$" && req.http.Cookie ~ "evolved_key") {
        hash_data(req.url);
    } else if (req.http.Cookie ~ "evolved_key") {
        hash_data(regsub(req.http.Cookie, ".*evolved_key=([^;]+).*", ""));
    }
    return (lookup);
}

sub vcl_backend_response {
    if (bereq.http.Cookie ~ "evolved_key") {
        unset beresp.http.Set-Cookie;
        if (bereq.url ~ "\.(css|js)$") {
            set beresp.http.Evolved-Type = "cssjs";
            if (beresp.ttl <= 0s) {
                set beresp.ttl = 120s;
            }
        } else if (bereq.url ~ "\.(png|gif|jpg|jpeg|swf)$") {
            set beresp.http.Evolved-Type = "image";
            if (beresp.ttl <= 0s) {
                set beresp.ttl = 120s;
            }
        } else {
            set beresp.http.Evolved-Type = "html";
            if (beresp.ttl <= 0s) {
                set beresp.ttl = 120s;
            }
        }
    }
    return (deliver);
}

sub vcl_deliver {
    unset resp.http.Evolved-Type;
}

Varnish 3

backend default {
    .host = "127.0.0.1";
    .port = "8080";
}

sub vcl_recv {
    if (req.request == "BAN_HTML") {
        ban("obj.http.Evolved-Type == html");
        error 200 "ban_html";
        return (error);
    } else if (req.request == "BAN_IMAGES") {
        ban("obj.http.Evolved-Type == image");
        error 200 "ban_images";
        return (error);
    } else if (req.request == "BAN_CSSJS") {
        ban("obj.http.Evolved-Type == cssjs");
        error 200 "ban_cssjs";
        return (error);
    } else if (req.request == "PURGE_SINGLE") {
        return (lookup);
    } else {
        if (req.url ~ "/form_key/" && req.http.Cookie ~ "evolved_formkey") {
            set req.url = regsub(req.url, "/form_key/[^/]*/", "/form_key/" + regsub(req.http.Cookie, ".*evolved_formkey=([^;]+).*", "") + "/");
        }
        if (req.restarts == 0) {
            if (req.http.x-forwarded-for) {
                set req.http.X-Forwarded-For =
                    req.http.X-Forwarded-For + ", " + client.ip;
            } else {
                set req.http.X-Forwarded-For = client.ip;
            }
        }
        if (req.http.Cookie !~ "evolved_key") {
            return (pass);
        }
        if (req.request != "GET" &&
            req.request != "HEAD" &&
            req.request != "PUT" &&
            req.request != "POST" &&
            req.request != "TRACE" &&
            req.request != "OPTIONS" &&
            req.request != "DELETE") {
            return (pipe);
        }
        if (req.request != "GET" && req.request != "HEAD") {
            return (pass);
        }
        if (req.http.Authorization) {
            return (pass);
        }
        return (lookup);
    }
}

sub vcl_hash {
    if (req.url ~ "\.(png|gif|jpg|jpeg|swf|css|js)$" && req.http.Cookie ~ "evolved_key") {
        hash_data(req.url);
    } else if (req.http.Cookie ~ "evolved_key") {
        hash_data(regsub(req.http.Cookie, ".*evolved_key=([^;]+).*", ""));
    }
    return (hash);
}

sub vcl_hit {
    if (req.request == "PURGE_SINGLE") {
        purge;
        error 200 "purge_single";
        return (error);
    }
}

sub vcl_fetch {
    if (req.http.Cookie ~ "evolved_key") {
        unset beresp.http.Set-Cookie;
        if (req.url ~ "\.(css|js)$") {
            set beresp.http.Evolved-Type = "cssjs";
            if (beresp.ttl <= 0s) {
                set beresp.ttl = 120s;
            }
        } else if (req.url ~ "\.(png|gif|jpg|jpeg|swf)$") {
            set beresp.http.Evolved-Type = "image";
            if (beresp.ttl <= 0s) {
                set beresp.ttl = 120s;
            }
        } else {
            set beresp.http.Evolved-Type = "html";
            if (beresp.ttl <= 0s) {
                set beresp.ttl = 120s;
            }
        }
    }
    return (deliver);
}

sub vcl_deliver {
    unset resp.http.Evolved-Type;
}

How do I know Varnish is serving cached content?

Use a tool like firebug to inspect the response headers from the server for an HTML, image, CSS or JS request - you will see an X-Varnish header containing numbers. if you see only one number string this indicates a cache miss and Varnish did not serve that content, if however you see two separate number strings then this indicates a cache hit and Varnish did serve that content.