The impersonator pattern is a testing architecture pattern. It deals with the problem of performing integration and functional tests over unstable, slow, not always available, data-changing or inexistent integration environments by providing an implementation which mimics the exact protocols and semantics of those environments while requiring minimal resources and providing full control over execution and managed data.
Testing against integration environments
Some classes of tests, such as integration tests, functional/acceptance tests and performance tests require testing against proper integration environments in order to validate system features. In practice, those environments have a combination of the following characteristics:
- They are unstable: their uptime is not guaranteed, or they may suffer with performance or behavior variances that can cause random timeouts and inconsistent responses
- They are slow: their response time is not fast enough to accommodate the demand required by automated testing; build scalability may demand more processing power than what integration servers provide
- They are not always available: they may not be running all the time, as the resource required to provide their services are too expensive or only available during specific times
- Their data is changing frequently: for any number of reasons the data required for testing may be changing due to external interactions and the cost of avoiding that may be prohibitive
- They may not even exist as teams may be working concurrently to build a solution that will be integrated at a later stage
In those cases, continuous integration can be seriously impaired. As tests would be frequently broken and their ability to pass won’t be completely dependent on the changes performed to the system, chasing problems may be difficult.
To mitigate the problem those types of environments pose to testing, a stability reference is required to allow for:
- Quick and scalable local builds
- Creating a reference for the data tests expect from the integration environment
- Proving the system works with reference data
- Pinpointing quickly what are the causes of failure for broken tests in the in the integration environment
The stability reference environment requires an extra stage for the build pipeline, which is completely standalone as it doesn’t depend on any external environment.
The stability reference
The following implementation approaches can be used to allow for integration and function testing outside the actual integration enviroment:
- Using test doubles, which implement the same programmatic interfaces as actual components in the system, but do not talk with external integration points. For example, a DAO may be reimplemented to provide static data and never talk with a real database
- Using servers that provide the same data, behavior and protocols as the actual servers in the integration environment. These servers may or may not be testing Fakes, as they may be suitable for production (eg. local deployments of the same servers used in the actual integration environment)
Test doubles require the creation of artifacts which contain embedded test code. This is not desirable, as it adds complexity to the build process and move the build pipeline away from a single-artifact discipline. Also, by not exercising the stack required to interact with external systems, test doubles add very limited value for integration and functional testing.
Servers that provide the same behavior and protocols as the actual integration environment are preferable for the stability reference. They may require the creation of custom software in order to replicate the same semantics as the integration servers, but they have the advantage of exercising the whole integration stack and do not require testing code to be built into artifacts (which is a good practice for build pipelines and testing in general).
These servers are loosely referred to as stubs, mocks, fakes, simulators, and so on, but in most cases these names do not help understanding what is the testing architecture in use. As a matter of fact, those names come from the unit testing space, in which compatibility is the primary goal in order to isolate and test behavior, whereas for integration and functional tests, compatibility, speed, reliability and non-intrusiveness are key.
To avoid naming ambiguities and establish specific semantics, those types of testing servers are going to be referred to as impersonators here. Also, it is assumed from here on that the stability reference is comprised by an impersonated environment.
Simply put, impersonators are testing servers that provide the same data, protocol and semantics as their integration environment counterparts. Impersonated environments (or standalone environments) are comprised exclusively of impersonators and do not depend on external servers. Impersonators may or may not run in the application space, but they are always externalized from the application code.
Impersonators can be implemented using one of the following strategies:
- Local deployments of the same software used in the integration environment, or a slimmed down version of that software (eg. using oracle express to impersonate an oracle database)
- Protocol compatible stock servers (eg. in-memory databases; SMTP servers, FTP servers, etc.)
- Home-grown [lightweight] servers, which can use a number of strategies to acquire data:
- Record-and-replay: using recording proxies or hooks into the application to create data flow snapshots as tests run over the integration environment (see self initializing fake for recording proxies)
- Fixed data: using a handcrafted test fixture, made by querying integration servers or otherwise
- Rules reimplementation: implementing rules to comply exactly with the semantics of the integration servers. A specific case of that are generators, which provide data using a known sequence.
Local deployments of production-compatible servers are sometimes convenient, but they require developers to install and configure servers in their machines (or depend on a single development sever, which imposes a single point of failure) in order to perform changes to the system. This may or may not be a problem, depending on the installation complexity and the amount of resources required to have those server performing well enough for testing. They do have the advantage of providing compatibility out-of-the-box.
Protocol-”compatible” stock servers generally run in memory and perform well, but may have compatibility problems which are hard to predict at the beginning of the development process. They are usually chosen to be used as in-memory servers, which do not require local installations.
Home-grown servers may be challenging to build and/or setup. They are usually not too complicated for read-only systems. However, transactional systems require state management, which require a careful strategy to differentiate and record transaction data. On the other hand, they are in full control of the developers, and are generally built to be lightweight.
An important directive for creating impersonated environments is that all impersonators have to have their own integration tests in order to guarantee they do behave as expected and remove the possibility a flaw in their implementations would cause the main application code to fail.
Use with continuous integration
If testing against integration environments is not guaranteed to work consistently, it is advisable to perform local builds against the standalone environment only, running all the tests against it (instead of using tricks such as smoke builds) as build scalability is achievable in that environment.
If the impersonated environment is properly implemented, builds against that environment should never break, which means there should be no excuse to have the stages in the pipeline associated with the standalone environment broken at any given point in time.
Testing against the actual integration environment is something that can be safely deferred to later stages in the pipeline. If tests are found to be broken there, one of the following possibilities should apply:
- The impersonators do not provide the same protocol and semantics as the integration servers
- Data has changed in the integration servers
- The integration servers were not available, or didn’t perform consistently
Throughout development, most broken builds would be related to reasons 2 and 3. The stability reference would provide means to identify the root causes for the broken tests and eliminate the need to look into the application code as a likely source of problems.
In order to check if the data provided by the impersonators match the data provided by the integration servers, it is convenient to implement a verify feature in each impersonator in order to validate their data and quickly identify if tests were broken due to data variances in the integration environment.
Standalone environments are sometimes useful for showcases, as they are reliable. Depending on the strategy used to implement impersonators, they can run in machines disconnected from the development environment and allow for off-site showcases.
If impersonators do not require local software installations, the costs and risks associated to development environments setups can be drastically minimized, as a single checkout of the codebase would be sufficient to allow for developing changes in the system.