I did a number of posts under the heading “You couldn’t write it” but in this case it had been so a different subject was required.
We’d been looking at an issue which had allegedly been fixed but the testing seemed to present some problems indicating it hadn’t worked.
As you would expect there a number of environments, almost all of which contain subsets of production, e.g. obscured data etc.
Sometimes however you need to test against an actual production data set, there may be many reasons for this and sometimes the situation of the requirements make it such that there no possible other option.
This was essentially one of those .. we ran the change into the database, ran the failed process and then ran the reports to check the results – only the reports didn’t display what was expected.
e.g. the generated data was not there.
Running the report code through SSMS , taking the application out of the testing, showed that in fact that the correct result set was being generated by SQL Server it just wasn't being displayed by the application. Checking the application event logs didn’t appear to show the report being run however.
So the reporting environment is SQL SSRS – what environment did the SSRS Server actually point at?
Not unsurprisingly and to much merriment it was discovered that the test SSRS environment had been changed to point at Production.
Thankfully this is a seriously restricted test environment and in actual fact the users also have access to Production .. so no serious harm there.
The test environment being used here is refreshed from Production every day so unless they were looking for data generated within or added to that environment the reports would have looked much the same, and presumably no-one had ever noticed, although in fairness it seemed no-one made much use of the application for testing reports - but that's another matter.
We did wonder if we should tell them that three years of testing had just been invalidated .. oh yes that was the rub the change was actually made three years ago, so all reports would have been run against production.
Just to add further injury one test environment pointed to a location we'd removed two years previously and for the final revelation our Training environment pointed to a non existent reporting server and had presumably been this way for two to three years.