Random Error While Building - "The process cannot access the file because it is being used by another process"

[jtwine]

So I have had this random error previously with older editions of IA. But I am getting them more and more often now and my installers are more complex than they were before. They also appear to happen at random points during the build process, usually closer to the start of the build. Images show examples of the build failing but progressing a little further each time.

BiuldFailure1.png (40.42 KiB) Viewed 3025 times

BiuldFailure2.png (42.38 KiB) Viewed 3025 times

BiuldFailure3.png (47.97 KiB) Viewed 3025 times

Running IA as an Administrator does nothing to change this behavior. (While the randomness of the errors might lead to a problem in IA itself, and I do not notice a rise in activity of the various anti-whatever stuff running on my machine, the remainder of this post assumes that IA is not the cause of the problem.)

Previously any posts on this forum regarding file access problems tends to be responded to with something along the lines of "check or disable your anti-virus/anti-malware/anti-DLP software."

Quite frankly, that is just not good enough anymore. Users may exist in an I.T. environment where they cannot simply "turn things off." Additionally, unless that is shown as a disclaimer before one purchases licenses for the product, it cannot be used as a catch-all excuse after purchase.

The software needs to be tolerant of temporary/occasional file access problems and work around them. Having to write a script to build an installer from the command line in a loop over and over again until it succeeds is a waste of time and should be unnecessary with modern software, especially when the support personnel are aware of a problem, and should have informed the developers by now.

However, being an experienced software developer, I am not going to simply complain about the problem and not offer possible solutions. So here are some suggestions:

1: Automatically retry all file accesses. This is the most important one - since the errors happen at random places, it stands to reason that retrying might help. Thus, all file access should be wrapped in such a way that automatic retries are tried after a small delay and then only give up once a certain number of file accesses have failed. Both the retry/wait interval and the number of retries should be configurable by the end user. Even Visual Studio retries 10 times before giving up - let this be a good example.

2: Ensure minimal file and sharing access is requested when opening files. In Windows, this has been a problem for as long as the CreateFile(...) function has been around. Make sure that if you are opening a file just to read its contents that you are specifying appropriate access and share values to maximize the chances of being able to open the file successfully. For example, do not specify GENERIC_ALL if you are just opening the file to read it. If you only need to read it, make sure your share flags do not include write access. In a tightly-controlled environment, maybe even specifying specific access flags instead of GENERIC_READ may be necessary. If it will work in a tightly-controlled environment, it will work elsewhere.

3: Show the user which process has the file locked! This is the ultimate and should be done if no other option is exercised. Many (many, MANY) moons ago I wrote a product called DeleteFXPFiles that would walk kernel objects in Windows XP to determine which process(es) held a file open that you were trying to delete. So I know it is possible to do this, and there is no real excuse not to. There are easier ways to do this today (e.g. the Restart Manager), and this should be implemented to help users determine the actual cause of a problem and then they have some ammunition to take to the I.T. department if necessary, or to help identify bugs or other unwanted behavior in IA or some other software.

I would hope that these suggestions would find their way to the development staff. These are not hard things to implement, and can only help improve the software.

We love IA here - but it is worthless if we cannot build consistently with it.

[FrancescoT]

Dear Jtwine,

I can understand all your considerations. However it seems to me that you are not considering that "Commercial AVs" are just applications and as any other application they may not work properly. Unfortunately when this happens they jeopardize the proper operation of other applications.

I'm aware of repeating myself once again, but Commercial AVs are not the keepers of the truth or the Bible.
The problem is instead that many people don't consider this and they totally rely on the fact that AVs are free of problems.

"Commercial AVs" should be NEVER installed on development machines or Production systems. I can assure you that there are many software vendors that already do this and most of them are big corporate companies.

I also understand of course, that Users may exist in an I.T. environment where they cannot simply "turn things off." This is totally true.
However this does not absolve the "Commercial AVs" from their responsibilities. Otherwise it's like putting in jail the victim instead of the murderer.

Frankly, we're already using many heuristics, some of which are trade secrets, to accelerate the build process.
The fact that the `complexity` of the project increases the issues has nothing to do with InstallAware.

At any rate and as I do Always, I will submit your suggestions to our development staff.

Regards

[jtwine]

I can understand all your considerations. However it seems to me that you are not considering that "Commercial AVs" are just applications and as any other application they may not work properly. Unfortunately when this happens they jeopardize the proper operation of other applications.

I never stated that the AV/AM/ADL applications were free of any problems. What I wrote was that a more robust implementation of certain things might be in order. The suggestions offered could even help locate lingering race conditions that might exist in IA. When someone says something like "Frankly, we're already using many heuristics, some of which are trade secrets, to accelerate the build process," it sounds like more places where something can go wrong. And it would follow that the more complex the project, the more these "heuristics" would come into play, and if any bugs exist, the more something is used, the more likely a problem will appear.

This is even more likely if these "heuristics" or any other tests are run near the start of the build, as opposed to being run throughout the entire build process, as that is where we are seeing problems. (After you get past the first 20-30 files or so, the chance of the error happening drops significantly.)

How about a way to disable any acceleration features - just a basic I-Can-Wait-15-Minutes-For-It build option.

And to hint at the complexity I referred to, the installer has two primary features, and deploys a set of identically named files that are from separate sources per feature, as well as a set of files that are feature independent. The reason I call this complex is because I can sometimes get the IA GUI into a state where these files and/or assemblies are nor getting displayed correctly in their respective GUIs. If the GUI can get messed up, ANYTHING can get messed up.

Lastly, I am not sure who told you that AV/AM software should "NEVER" be installed on development or production systems, but it sounds like they have never experienced a virus or malware situation that could infect newly built executables or do something like affect the operation of a web site or infect downloads obtained from it. That is like taking "you do not need to run backups" advice from someone that has never experienced a data loss event, or taking lane-changing suggestions from someone that never drove a car before.

[FrancescoT]

Dear Jtwine,

probably there was a misunderstanding, I never said to not use an AV. I said instead and I repeat once again, that "Commercial AVs" should be NEVER installed on development machines or Production systems. The most effective alternative is to use the Microsoft AV/AM that comes now included with any recent Microsoft OS ...and which is also free. I'm primarily a developer and for about 20 years I also used a "Commercial AV" as many Others. Then one day of 4 years ago my production server went mad ...I leave you to imagine the cause.

For the rest, I will certainly submit all your suggestions to our development staff.

Regards.