Working Is Not the Same as Recovering From Failure
A demonstration proves that one path through your application works when everything goes right. It says very little about what happens when something goes wrong — and in production, something eventually goes wrong.
Think of a card payment at a cafe. The happy path is simple: tap, approve, receipt. What makes payment systems trustworthy is everything else: what happens when the connection drops mid-transaction, when the same card is tapped twice, when the till software is updated overnight. The quality of a system lives in how it behaves at exactly the moments nobody is watching.
To be clear: this is not an AI problem. Human-written software fails in precisely the same places, and experienced teams have shipped every one of these defects. The difference with rapid AI-assisted development is usually not worse code — it is that nobody has verified the failure behaviour yet. Unverified is the key word.
Where Customer Data Actually Gets Into Trouble
In plain terms, the problems that hurt a business look like this: missing records, duplicate payments, balances that do not add up, encrypted data that can no longer be read, and upgrades that quietly damage what was already there. Behind those symptoms sit five technical risk areas:
Encryption keys and secrets. Your app may encrypt customer data with a key that exists on your laptop but is configured differently in production. If a key is missing or changes, does the app refuse safely — or fail in a way that loses access to the data?
Partial writes. Saving one customer action often means several database changes together. If the operation fails halfway, you can be left with half a record — an invoice with no line items, a payment recorded but a balance not updated.
Schema migrations. A database upgrade that works perfectly on an empty development database can behave very differently against thousands of existing customer rows.
Concurrency. Two people pressing save at the same moment. Without deliberate handling, one update can silently overwrite the other, or duplicates can appear.
Development versus production drift. Security and data behaviour often depend on configuration, permissions and environment settings that change between your machine and the live server.
None of this means your app has these defects. It means that until the behaviour is exercised deliberately, nobody knows — including the AI that wrote the code.
Follow One Customer Record Through the App
A typical AI-built application has a simple data path. The numbered points are the places where, in any application, a working system can quietly lose data integrity.
The numbered markers are possible failure locations found in applications of every origin — not confirmed defects in yours. A targeted assessment selects up to three agreed tests. It does not test every risk shown.
Five Questions to Ask Before Going Live
You can run this self-check without any external help. For each question, the honest answers are "yes, verified", "no", or "I don't know".
- If the encryption key is missing or changes, does the app fail safely without overwriting data?
- If an operation fails halfway through, can it leave a customer record partially updated?
- If two people change the same record simultaneously, can one update silently erase the other?
- Can a database upgrade run safely against existing customer records, not just an empty development database?
- Does production behave correctly when configuration, permissions, secrets or dependencies differ from development?
If you cannot answer a question, that does not mean your app has a defect. It means the behaviour is unverified — and for an app holding real customer data, unverified is the thing worth fixing first.
What a Targeted Failure Test Actually Looks Like
A failure test is not a code review and not a scan. It is a controlled experiment with a defined starting state, one deliberately injected failure, a stated expectation, and an observed result. Here is the shape of one, using synthetic data:
Illustrative example with synthetic data — not a real customer incident.
Starting state: a test database with one synthetic customer. Creating an invoice is a three-part operation: insert the invoice, insert its line items, update the customer's running balance.
Injected failure: the process is interrupted after the invoice row is written but before the line items and balance update complete — simulating a crash, timeout or lost connection at the worst moment.
Expected behaviour: all three changes succeed together, or none of them do. The database should never hold an invoice with no line items and an untouched balance.
Observed result: whatever the database actually contains after the interruption — recorded exactly, whether the app passed or failed.
Evidence: the test script, the database state before and after, and reproduction steps a developer can run again independently.
The same pattern applies to a missing encryption key, a migration against existing rows, or two simultaneous updates. The output is not an opinion about code quality — it is observed behaviour, documented so that any engineer can check it.
This kind of testing is exactly what SummitWright's AI Application Data Integrity Assessment performs on one critical workflow of your application.
What Testing Can and Cannot Establish
Honesty matters more here than anywhere else. A passing test establishes behaviour under the tested conditions — nothing more. It does not certify the whole application, prove it secure, or guarantee that untested paths behave well. A targeted assessment is also not a penetration test, a full security audit, or a production-readiness certification.
It is also entirely possible for an assessment to find no defects in its tested scenarios. That is a legitimate, useful result: it converts "I don't know" into documented evidence about the behaviour that matters most.
And a word on thresholds: if your app is an experiment with no real customer data, you probably do not need any of this yet. Testing becomes economically rational when the app holds customer records, takes payments, performs irreversible business actions, stores legally significant information — or is about to launch to real users.
The AI Application Data Integrity Assessment
Fixed-scope assessment — A$1,650
One qualifying application, one critical data workflow, up to three targeted executable tests agreed with you in advance, a written evidence report, and a technical walkthrough of the findings. Delivered directly by Al Elmi in an approved non-production environment — your source code and customer data stay under your control.
No GST charged — SummitWright is not registered for GST.
Explore the A$1,650 Data Integrity AssessmentThe assessment investigates a defined set of behaviours and records the results. It is not a comprehensive security audit, and other defects may remain outside the tested scope.
What Happens if a Test Finds a Problem?
Every finding is documented with the observed behaviour, supporting evidence, reproduction steps and a prioritised recommendation. The report is written to stand alone: it stays useful whether you fix the issues yourself, ask your own developer, or engage SummitWright.
If you do want help with fixes, remediation is a separate, separately authorised engagement at A$1,250 per day — it is never bundled into the assessment, and the assessment fee is not credited against it. That separation keeps the report honest: it is evidence, not a sales pitch for repair work.
Not Sure Whether Your App Needs This Yet?
A good fit: an AI-built or AI-assisted app that handles real customer records, payments or important business data — with source-code access and a safe place to test. Not a fit: a visual prototype with no meaningful data, or an emergency production incident needing immediate repair.
A free 20-minute suitability call establishes whether a meaningful test is possible for your app. It is a scoping conversation, not a technical audit — no repository upload, no sensitive data.
Book a free 20-minute suitability callDo not submit source code, passwords, API keys or customer records through the contact form.