PB 10.04 and random memory error

[Patrice Terrier]

More and more people are experiencing memory error problems with the latest version of the PowerBASIC compiler (10.04).

The main symptom, is that the application doesn't fire when you run it, or may fire only 1 out of 3/4 or more.

This is related to a random memory error that occures {only} in version 10.04.

In my case it is related to the use of global dynamic array altogether with ARRAY SCAN, like in the code below:

FUNCTION skMenuSkinItemOk (BYVAL nID AS LONG) AS LONG '// 03-21-2016
    LOCAL K, nItem AS LONG
    'IF UBOUND(mSkin) > 0 THEN ARRAY SCAN mSkin(), = nID, TO nItem
    for K = lbound(mSkin()) to ubound(mSkin())
        if mSkin(K) = nID then nItem = K: exit for
    next
    FUNCTION = nItem
END FUNCTION

Replacing the line in red with the line in blue solved the problem i had in WinLIFT when compiling with 10.04.

I have attached a zip file, with a short demo that is able to reproduce the problem every time (this demo was also sent to the beta).

Note: This problem seems to have occured since the introduction of PowerArray.

...

[Frederick J. Harris]

I doubt if you recall Patrice, but a number of years ago (actually, I do know when it was - winter 2013), I had a severe problem with my COM based grid control.  It only occurred on Windows XP.  I'm pretty sure it never occurred on Win 7.  The prerequisites for it occurring were Windows XP and PB 10.04.  Not sure about 10.03.  But on my Win XP I kept 10.02 as the last 'safe' version, and to this day I use that as my 'golden master' so to speak when compiling the grid.   Where the error would get started was with the Parse statement and dynamically allocated and freed arrays.

[Patrice Terrier]

You said 10.02 is working, did you try it with my demo example, and what is the result?

...

[Frederick J. Harris]

I just downloaded your demo now and tried it Patrice.  I only tested so far on XP sp3.  The Demo seemed to run every time I started the exe.  Two windows showed up; a large blank one, and a small yellow one with presumably debug info in it.  Quite a few lines said something to the effect "by me error here" or some such.  I started it about five times and it worked every time.  Didn't experience any crashes or failures to start.

Next I deleted the exe and compiled with PB 10.02 on the XP laptop and I immediately got the error about can't compile because exe is running.  So I opened Task Manager and had to close out about five exes or so.  I wasn't aware at the time that the demo wasn't closing out properly.  So after that I ran the version I compiled with 10.02, and that ran fine too. 

Do I need to go into the code and find the code you referred to and change it to get a failure Patrice?  I guess what I'm saying is that I wasn't able to determine anything from my end, as both your download and what I compile with 10.02 are working the same.  They both run but fail to terminate correctly??? 

[Frederick J. Harris]

I just looked at my newer Windows 7 machine and I see I have 10.02 on there too.  I thought maybe I put 10.04 on it but apparently not.  That experience I had several years back was likely the most harrowing experience I ever had with attempting to find the source of a bug.  Left pretty deep scars.  I guess that's why I don't run 10.04 even on post xp machines, even though on those I've never experienced the bug.  However, I'd be willing to upgrade it to 10.04 and test if you like.  I can always uninstall and build 10.02 back from the initial 10.0 release.

[Carlo Pagani]

Crashed in 10.00, 10.02 and 10.04 for me all in the same place as you expected.

Having said that - they all work in Debug Mode?

Process Monitor shows a buffer overflow.

Your issue may stem from https://msdn.microsoft.com/en-us/library/ff545817.aspx

[Patrice Terrier]

The one i keep using myself for GDImage and WinLIFT is PB 9.05, this version produces smaller code, and it has always worked fine by me.

Use PB 9.05 to recompile the provided WinLIFT.bas and Main.bas, then run the resulting ObjReader.exe, and it should go to full completion.

The memory management with the debug version couldn't be compared with the memory management of the real compiled code.
This is the reason why i have never used the debug version, but only my zTrace.dll to be like in real EXE configuration.

Of course, none of the 10's additions could be used with PB 9.05, nor the latest José Roca's include files, however for plain SDK code this is not a problem

I have already reported this problem to Robert Zale before his passing, but was not able to replicate the random memory error, until this demo that occures always at the same place by me.

I doubt this problem will ever be solved, thus i shall keep using 9.05 for my PB's 32-bit ANSI code production, and switch to C++ for everything else, no more risk to rely on the work of a single man for the choice of a compiler.



...

[Carlo Pagani]

I had only compiled main.bas.

Once I compiled WinLift.DLL in 10.02 it worked fine. Seems Fred is right 10.02 is most stable version.

Now more confused - Works in 10.04 if I compile both DLL and EXE.
What version did you use in this upload Patrice?

[Frederick J. Harris]

I had only compiled main.bas.

That's all I had done too.   But I just compiled the dll and nothing changed.  It starts every time (compiled with 10.02) , the only abnormality being I have to close it out with Task Manager.  I'm assuming its stripped down code and the original doesn't work like that. 

I'm not certain this issue you have had Patrice is the same problem I had with my grid, but it seems at least possible if not likely. 

The determination I made about the 10.02 - 10.04 versions is not something I was ever happy about.  The only fact I was able to arrive at with complete certainty after something like three months of miserable work trying to solve the issue I was having was that if I compiled my grid control with 10.04 it would fail in many but not all cases.  The failures sometimes (but not often) involved corruption of data in the grid, but what happened 100% of the time was a crash at program termination.  And this only occurred on 2000/XP machines - never on Win7.  When the diasssembly was viewed in the Visual Studio debugger it showed an exception in nt.dll.   This is just a wild guess on my part but I suspected a possible cause being memory being de-allocated twice.  That's just a wild guess.

These types of bugs are horrible in the extreme.  In my case I poured over my code for months testing every conceivable possibility of memory corruption due to an error on my part.  I simply couldn't find anything.  I was at my wits end.  I knew there was a problem but I just couldn't find it.  In desperation I figurred I had to come up with some way of attempting to isolate where the bug was at so I wouldn't have so much code to study and test.  When I was working on the grid I first started out with it as a custom control.  I had something like fifteen folders of code each with a different version as I built the thing up.  Then I converted the code over to COM architecture and continued to build it up by adding additional features.  I ended up with eight versions each in a seperate folder.

So what I decided to do to try to isolate the bug was start back from my finished version, which was version 8, and keep working backwards through the versions until I found a version which didn't evidence the bug.  When I found such a version, I reasoned the bug I was looking for would have to be in some feature added after that version, and I could thereby reduce the quantity of code I would have to test.  When I got back to version 4, I finally hit a version that didn't crash on termination with the data set I had found that would cause a crash every time.  Naturally, this made me very happy.  I thought, "Maybe now I can solve this horrible thing that was ruining my life!".

But before I continue, let me interject something.  My development of my grid control was way back shortly after Bob released version 10 of his Windows compiler.  I'm not sure of the exact date.  But I immediately bought version 10.00 as soon as he released it.  I was most interested in native UNICODE above everything else.  But I couldn't start my grid project with 10.0 because there were some bugs in it.  Specifically, Parse() didn't work with unicode strings, and several other string functions I had found.  I immediately sent in a bug report to Bob with supporting code and he fixed it almost immediately and provided me a personal link to the fixed compiler which I then downloaded and used.  I believe he added me to the beta testers team at that point.  But in any case, within a week or two he posted update 10.01 for everyone, which contained the fixes.   So that was actually the version I was using when I developed the grid.

Now, getting back to my story on version 4 of the grid, I thought I had finally found a version without my bug, whatever it was, because when I ran the old dll binary that had been compiled at the time several years back when I had been developing the grid it didn't crash on the test data that always crashed.  Remember, at this point I was still laboring under the assumption that I had made a coding error somewhere, and I was sparing no effort in my attempt to find it.  But then I found out something amazing.  If I re-compiled that old code back in version 4 --- it crashed!!!!   

So here's what I had.  I had the several year old dll compiled with either 10.01 or 10.02 that didn't crash at termination, but if I recompiled with 10.04, which I dutifully upgraded to when Bob provided it, it would crash!

So now I finally was on the track of solving this issue.  But the solution wasn't something I was really happy about.  I was hoping to find the error in my code which I could fix.  If this was finally the issue, I couldn't fix it.  And to make matters infinitely worse, Bob had passed away!

So there you have about the full story on this in terms of how I made the determination concerning the 10.02 thing.  It wasn't something lightly made.  I worked on this issue for something like three months trying to solve it.  I'm a bit embaressed to have to admit that.  Doesn't say much for my debugging skills.  But that's the sorry truth of it.

And again, I don't know if this is related to the issues others might be having, or to Patrice's issue.  And I have to say I never ran into this issue anywhere else in any other coding I had done with 10.04 before I abandoned it.  But because of it, I did drop back to 10.02.  At this point several years later my memory is a bit dim as to why I didn't use 10.03.  I don't recall if that version evidenced the bug or not with my grid.  I'd have to re-check, but I'm not of a mind to bother.  I believe the bottom line for me was that everything i personally needed worked perfectly with 10.02, so I just decided to stick with that.  Anyway, my impression was that after 10.01 or 10.02 all Bob worked on was DDT stuff I didn't use.     

[Frederick J. Harris]

One other thing I didn't mention above.  In my months of work on this, one of the things I did was tear out all the complex and voluminous code in the grid - possibly what Patrice did above with his ObjReader code, in an attempt to isolate the lines causing the failures.  Where it was ocurring was with the Parse statement in the WM_CREATE handler code which creates the grid.  Of course, this involves dynamic memory allocation of arrays, which seems to be the common link in cases of this bug rearing its ugly head.

[Patrice Terrier]

What version did you use in this upload Patrice?

Oops, i may have used the same ZIP that i sent to the beta to show the error (aka version 11).

With version 10.04 the program stay in memory, and you have to use the task manager to get rid of it.
(probably because the zTrace process is still in memory, even if you close the trace window).

zTrace was just used to show the exact location where the memory error occures inside of the WinLIFT DLL, while using menu recursion and the function skMenuSkinItemOk, to give the beta something they could use to solve the problem.

...