You’re Using Transients Wrong
23rd of October
The Transients API is an incredibly useful API in WordPress, and unfortunately one of the most misunderstood. I’ve seen almost everyone misunderstand what transients are used for and how they work, so I’m here to dispel those myths.
For those not familiar with transients, here’s a quick rundown of how they work. You use a very simple API in WordPress that acts basically as a key-value store, with an expiration. After the expiration time, the entry will be invalidated and the transient will be deleted from the transient store. Transients essentially operate the same as options, but with an additional expiration field.
By default in WordPress, transients are actually powered by the same backend as
options. Internally, when you set a transient (say
foo), it gets transparently
translated to two options: one for the transient data (
_transient_foo) and an
additional one for the expiration (
_transient_timeout_foo). Once requested,
this will then be stored in the internal object cache and subsequent accesses in
the same request will reuse the value, in much the same way options are cached.
One of the most powerful parts of the transient API is that it uses the object
cache, allowing a full key-value store to be used in the backend. However the
default implementation, and how the object cache can change this, is where two
major incorrect assumptions come from.
Object Caching and the Database
The first incorrect assumption that developers make is to assume the database will always be the canonical store of transient data. One big issue here is attempting to directly manipulate transient data via the option API; after all, transients are just a special type of option, right?
In the real world however, anything past your basic site will use an object cache backend. Popular choices here include APC (including the new APCu) and Memcache, which both cache objects in memory, not the database. With these backends, using the option API will return invalid or no data, as the data is never stored in the database.1
I’ve seen this used in real world plugins to determine if a transient is about
to expire by directly reading
_transient_timeout_foo. This will break and
cause the transient to always be counted as expired with a non-default cache.
Before you think about how to do this in a cross-backend compatible way: you
can’t. Some backends simply can’t do this, and until WordPress decides to
provide an API for this, you can’t predict internal behaviour of the backends.
The second incorrect assumption that most developers make is that the expiration date is when the transient will expire. In fact, the inline documentation even states that the parameter specifies the “time until expiration in seconds”. This assumption is correct for the built-in data store: WordPress only invalidates transients when attempting to read them (which has lead to garbage collection problems in the past). However, this is not guaranteed for other backends.
As I noted previously, transients use the object cache for non-default implementations. The really important part to note here is that the object cache is a cache, and absolutely not a data store. What this means is that the expiration is a maximum age, not a minimum or set point.
One place this can happen easily is with Memcache set in Least Recently Used (LRU) mode. In this mode, Memcache will automatically discard entries that haven’t been accessed recently when it needs room for new entries. This means less frequently accessed data (such as that used by cron data) can be discarded before it expires.
What the transient API does guarantee is that the data will not exist past the expiration time. If I set a transient to expire in 24 hours, and then attempt to access it in 25 hours time, I know that it will have expired. On the other hand, I could access it in 5 seconds in a different request and find that it has already expired.
Real world issues are common with the misunderstanding of expiration times. For WordPress 3.7, it was proposed to wipe all transients on upgrade for performance reasons. Although this eventually was changed to just expired transients, it revealed that many developers expect that data will exist until the expiration. As a concrete example of this, WooCommerce Subscriptions originally used transients for payment-related locking. Eventually, Brent (the lead developer) found that these locks were being silently dropped and users could in fact be double-billed in some cases. This is not a theoretical issue, but a real-world instance of the expiration age issue. The solution to this particular issue was to swap it out for options, which are guaranteed to not be dropped.
When Should I Use Transients?
“This all sounds pretty doom and gloom, Ryan, but surely transients have a valid use?”, you say. Correct, astute reader, they’re a powerful tool in the right circumstances and a much simpler API than others.
Transients are perfect for caching any sort of data that should be persisted across requests. By default, WordPress’ built-in object cache uses global state in the request to cache any data, making it useless for caching persistent data. Transients fill the gap here, by using the object cache if available and falling back to database storage if you have a non-persistent cache.
One application of this persistence caching that fits perfectly is fragment caching. Fragment caching applies full page caching techniques (like object caching) to individual components of your page, such as a sidebar or a specific post’s content. Mark Jaquith’s popular implemention previously eschewed transients due to the lack of garbage collection combined with key-based caching, however this is not a concern with the upcoming WordPress 3.7.
Another useful application of transient storage is for caching long-running tasks. Tasks like update checking involve remote server calls, which can be costly both in terms of time and bandwidth, so caching these makes sense. WordPress internally caches the result from update checking, ensuring that excess calls to the internal update check procedures don’t cause excessive load on the WordPress.org server. While the object caching API would work here, the default implementation would never cache the result persistently.
Transients are awesome, but there are some important things to watch out for:
- Transients are a type of cache, not data storage
- Transients aren’t always stored in the database, nor as options
- Transients have a maximum age, not a guaranteed expiration
- Transients can disappear at any time, and you cannot predict when this will occur
Now, go out and start caching your transient data!
- The reason I say invalid or no data here is because it’s possible for a transient to be stored in the database before enabling an object cache, so that would be read directly. [↩]