Orthogonal is a math term, borrowed for Computer Science. It only makes sense in talking about a value in relationship to other values.
An orthogonal value can change without the other variables changing. In math, this feature indicates it is independent of the other values. Sometimes people use this independence to indicate it is a measurement in a different "dimension" than the other items.
The 12 factor use of orthogonal is borrowing from these ideas, they clearly want something that can change independently of the other items. This means a few things:
Values that represent the same thing cannot be duplicated, such that updating one requires the update of another.
Values that have more than one presentation (like domain names vs ip addresses) should have only presentation within the configuration approach. It might be ok to use an IP address or a host name, but you don't include both.
These ideas are easy to implement for "atomic" values, where one value is clearly independent of the others; however, these ideas are harder to implement for "compound" values, where one value is part of another value. For example, a JDBC database connection string can be represented in two ways; as a single string or as a compound collection of protocol, database driver id, hostname, port, database name, connection options, etc.
The latter way makes it easy to change the hostname; but, when you decide to change the database driver id, you notice that the default port will likely have to change to match the database driver id. The default port is not orthogonal to the database driver id. Yes, the port (if specified) can be changed independently, but the update pattern is such that nearly every time one needs to change the database driver id, the port will also have to be verified.
Assuming one took the compositional approach, this means the driver id and the port are not orthogonal settings. If one took the connection string approach, then there is only one setting, so it is by definition independent and orthogonal.
Now, assuming you took the connection string approach. The database driver (was) typically loaded by class name, which isn't embedded in the connection URL (string). If you specified the class name in a different environmental variable, the two are clearly not orthogonal. This means you cannot choose the single string JDBC_URL and a DRIVER_NAME set of variables and strictly adhere to "orthogonal" configuration. Thankfully, there are other approaches.
One can pre-parse the JDBC connection URL, pulling out the driver id, and then use a lookup table to find the class name, and preload the class. This would remove the need for two non-orthogonal settings, at the cost of having a bit of convenience utility code.