The error message was: Cannot read property 'id' of undefined. Standard JavaScript null reference. The kind of error you've seen a hundred times. You know what it means. You start looking for the undefined object.
Three hours later you find it. The object is undefined because a database query returned no results. The query returned no results because it was filtering on a field called tenant_id. The tenant_id was coming from an environment variable. The environment variable was not set.
Not missing from the .env file. Not missing from the deployment config. Set to an empty string. Somewhere in the deployment pipeline, the variable had been set to "" instead of the actual value. The application read it, treated it as a valid string, passed it to the query, got zero results, and then crashed when it tried to use the result.
The opened terminal team has a running list of debugging sessions that traced back to environment variables. It's longer than you'd expect. The pattern is always the same: the variable exists, the application reads it, the value is wrong in a way that doesn't immediately surface as an error. Empty string instead of null. Wrong format. Trailing whitespace. A value that was correct in one environment and wrong in another because someone copied the wrong config.
The fix for this specific case was a startup validation check. Before the application does anything else, it reads all required environment variables and validates them — not just that they exist, but that they match expected formats. If validation fails, the application refuses to start and logs exactly which variable is wrong and why. It's a five-minute addition that has saved hours of debugging.
The broader lesson, which I keep relearning, is that environment variables are a trust boundary. You're trusting that the deployment system set them correctly. That trust is often misplaced. The calitoy core post about configuration validation makes this point better than I can — the argument is that any value that comes from outside your application should be validated at the boundary, not assumed to be correct.
There's a related problem with variable naming. We had a case where two services both used a variable called DATABASE_URL but expected different formats. One expected a connection string. One expected a host:port pair. Both were set correctly for their respective services. When someone copied a config between services, the wrong format ended up in the wrong place. The application started fine. The error only appeared on the first database query.
The tarosyn dev team has moved to prefixed variable names — every variable is prefixed with the service name — which eliminates the copy-paste confusion. It's more verbose but it's never ambiguous. Worth the tradeoff.
More on environment variable management at risen today — there's a piece there about validation patterns that's practical and not preachy about it.