0

Microsoft releases upgrades and changes to .net, msbuild and Visual Studio quite frequently.

How can I be sure the MSIL code created by msbuild or the Roslyn compiler in VS 2015 will be the same or behave the same after the updates?

Completely testing an application after every Microsoft update would be time prohibitive.

How can I be sure my applications won't be affected? Is it safe to assume that Microsoft won't push changes that will affect the compilation of existing code?

EagleHail
  • 3
  • 1
  • 1
    http://stackoverflow.com/a/35731566/128384 covers it pretty well - that being said, you could as well ask 'how can I be sure my applications won't be affected by the next bios/os/... update'? You can't, this is software. – stijn Mar 02 '16 at 07:45
  • If testing takes too much time, you're probably doing it wrong. And not testing, or not upgrading, would probably cost even more time at some point. That said, chances are really tiny that any Microsoft update will break you code specifically. There's quite some focus brought on backward compatibility and heavy testing going on before any release, I can't recall of an update breaking anything in any piece of code I ever made. – Laurent S. Mar 02 '16 at 09:23

2 Answers2

8

Completely testing an application will be time prohibitive?

Write some unit tests. Write some integrations tests. Automation is your friend. It should take under a minute to provide 90% or so confidence that your app is working properly. Then it should take an hour or two to get another 5% or so.

That said, .net has been around for nearing two decades now - and how many bugs can you recall? There was the TCO bug recently. There was some behavioral changes to sorting in 4.0... And that's it?

Every single coder on your team writes more bugs in a day than the .NET teams have released in the last year.

So yes, there are a few dozen things you should worry about before Microsoft's compiler updates.

Telastyn
  • 108,850
  • 29
  • 239
  • 365
  • 1
    It may also be worth mentioning that the .Net compilers are non-deterministic and have been for as long as I can recall So the same compiler, source etc running on the same machine is likely to yield subtly different results for each compilation. – James Snell Mar 02 '16 at 13:11
2

The way this works for critical software is to certify it with particular systems and versions. You do not update your dev environment unless you are prepared to re-certify the software. This then means running through all your tests to verify that the product still works perfectly.

How much time this takes, and how often you do it depends on the software product - some device are still running on XP for example, they're not networked and don't need to be updated so they don't get updated. The problem with this is that re-certifying it takes a prohibitive amount of time so when an update it required, it becomes much more difficult.

The same applies for different hardware - for example, my current product uses video cameras, we had a new model from the same manufacturer and it simply failed to work reliably, so we had to work out why and update the software with an updated stack.

gbjbaanb
  • 48,354
  • 6
  • 102
  • 172