Technical support assessment: an incident drill for your online store

Technical support assessment should show whether the team can detect a failure, take responsibility and return the store to an agreed working state. A controlled incident drill is more useful for this than a list of tools in a commercial proposal. Check the entire path: alert, support request, decision, recovery and confirmation from the business.

This approach is intended for the manager of an online store that already has support and wants to check its readiness for problems. This is not about accepting a new website or testing protection against attacks, but about checking the response process without harming customers.

What an incident drill actually demonstrates

An incident drill is an agreed failure scenario with defined boundaries, participants and completion conditions. For example, in a test environment the store accepts an order, but the back-office system does not confirm it. The team must distinguish a page loading successfully from a business operation completing successfully, identify the person responsible and verify the outcome of the fix.

The Google SRE guide to incident response discusses resolving the fault and coordinating the team's work separately. For a store, this distinction has practical value: a developer looks for the cause, a coordinator aligns actions, and a business representative tells managers what they can promise customers. In a small team, roles can be combined, but none should be left unassigned.

Choose a scenario based on its impact on sales

Start not with the most frightening technical event, but with a process whose interruption the business needs to notice quickly. Checkout, order transfer, confirmation delivery or a manager's access to their work are suitable options. The scenario should include an initial state, an expected symptom and evidence of a return to normal. These three elements make the result reproducible.

These are training options, not problems from a specific project. For payments, use the provider's agreed test mode. The drill must not cause real charges, shipments or messages to customers.

Agree on boundaries and responsibilities before starting

Before the drill, you need a short document, not just a chat message. Record what is being checked, the available environment, permitted actions, the person responsible for stopping the drill and the people who accept the result. Add contacts for the hosting provider, developer and integration owner if they are involved. Check that these participants have agreed to take part and are available during the selected window.

Response is not the same as recovery

Support response time is not the time needed to restore sales. An automatic reply confirms receipt of a request, not the start of diagnosis. For the drill, record separately when the alert is received, when the responsible person acknowledges it, when action begins and when recovery is verified. This shows which transition cost the team time, without arguments over a single overall metric.

If the contract includes an SLA—a service-level agreement—check its priority definitions, working hours and timing rules. A whole store being unavailable and an error in a single information block may follow different paths. Do not assume a contractor provides round-the-clock response merely because the monitoring system collects data around the clock.

A designated person makes decisions with consequences

Agree separately on who may suspend data exchange, restrict checkout or authorize data recovery. The technical ability to perform an action does not mean permission to do so. The scenario should also name a backup decision-maker in case the primary person is unavailable. This is an organizational check; passwords, tokens and personal data must not be included in the drill record.

Prepare the test environment and check observability

The test environment should reproduce the required path without interacting with real customers. Isolate email delivery, payments, shipping and data exchange with production systems. Drill data must be synthetic or properly anonymized. Before starting, the responsible technical specialist confirms the isolation and the ability to return the test instance to its initial state.

Check more than homepage availability. The Google SRE monitoring recommendations distinguish internal metrics from checks of user-visible behavior. For a store, a server response does not yet confirm that an order has been saved and received by a manager. Combine a verification scenario with the log of the corresponding operation.

For each alert, define a recipient, a fallback channel and the expected action. A message saying “there is an error” without naming the environment and the impact on the process creates unnecessary follow-up questions. At the same time, do not forward complete order details to a general chat. An agreed test identifier, event time and link to the system where an authorized specialist can check the details are sufficient.

Before simulating the failure, complete a successful baseline run. Otherwise, you will not know whether the symptom was caused by the drill or the test environment was already faulty. Save the expected statuses and the result in the back-office system. If the baseline scenario does not work, postpone the drill and resolve this uncertainty first.

Test the new response procedure before putting it into use

A new support procedure should be tried out before relying on it during a real failure. During the run, one participant keeps an event log. They record actual transitions, approval requests and obstacles, rather than judging people during diagnosis. All timestamps must use an agreed time zone.

1. Confirm the initial state, test data and permission to start. Name the person who can stop the drill immediately.
2. Reproduce one agreed symptom strictly within the permitted boundaries. Record when it appears, not just when the support request is sent.
3. Check alert delivery and assignment of responsibility. If the alert does not arrive, record this as a result rather than hiding it with a manual message.
4. Work through diagnosis and approval of the next action. Ask the team to explain which process is disrupted and what they will check next.
5. Perform the agreed recovery on the test instance. Do not change the production system just to complete the drill successfully.
6. Repeat the verification operation and compare the result across all systems involved. Return the test instance to the agreed state and check that no unintended test actions remain.

Hypothetical example: an order appears in the store, but there is no confirmation from the back-office system. Clicking the data exchange button again is not, in itself, a success. You need to find the record, check the items and quantities and ensure that no duplicate has been created. This illustrates an acceptance criterion, not a real GL.ua case.

Check recovery, not merely the existence of an archive

Website recovery testing should end with a working environment, not a message saying that extraction was successful. If this scenario is part of the drill, the team selects a backup in advance and records what it contains. For Drupal, the official backup guidance covers the database, user-uploaded files and the ability to restore the corresponding code state.

On an isolated test instance, check that the catalog and required files open, that an authorized user can sign in and that the verification operation works. Record the point in time to which the data has been restored and what was not included in the backup. Agree on the acceptable recovery duration and acceptable loss of newer data: these are different requirements, and more frequent backups do not necessarily make deployment faster.

Restoring the entire database to an earlier point may remove new orders or status changes. A successful drill therefore does not automatically authorize the same action on the production store. Real recovery requires a separately agreed plan for reconciling more recent operations. If there is no such plan, record the limitation rather than calling the system fully ready.

How to assess the outcome, schedule and cost

The outcome of the drill is a report with evidence and specific changes to the procedure. For each gap, record an owner, the next action and a method for retesting. For example, “no access to the log” becomes approved provision of the required access followed by another test. “Improve communication” does not resolve the problem unless a new rule is defined.

On the second run, the team already knows the scenario: this tests the instructions, rather than proving the same speed for a different failure. Record time spent waiting for access, third-party services and approvals separately, but do not remove it from the overall downtime.

What determines the duration

The plan includes environment preparation, coordination with participants, the run itself, a debrief and retesting the fixes. Database size, documentation quality, the number of integrations and the availability of external teams affect the schedule. Estimate these components separately. If there is no test environment, creating one is a separate task, not a hidden part of a short support demonstration.

After a significant change to an integration or on-call arrangements, retest the changed path. The business agrees the frequency of full drills with the team according to the risks.

Which costs to calculate separately

The estimate includes scenario analysis, test data preparation, the work of technical specialists and the coordinator, test environment resources and backup storage. Paid monitoring tools and participation by integration vendors may be additional costs. Clarify whether the debrief and another run are included in the support service. Estimate remediation of the technical debt discovered separately; otherwise, the price of the drill will conceal the cost of changes.

When to stop and what to do next

Stop the drill if the test affects real customers, sends production messages or goes beyond the agreed boundaries. Do not start it during a real incident or a critical sales campaign. If there is no safe test environment, walk through the scenario without technical changes: this will check contacts and decisions, but will not prove that recovery works.

A successful run demonstrates readiness for the agreed scenario under the tested conditions, not the absence of all future failures. What matters is a reproducible procedure: who receives the alert, who acts and who confirms the outcome.

Discuss website technical support with GL.ua (Glyanets): prepare a list of critical operations, existing response rules and questions for the assessment. This will help define the drill's boundaries and the required outcome.