Sign Up
VIPM › Posts › Store File Continue Not Working Correctly

suggestion

Store File Continue Not Working Correctly

Suggestion shared 5 months, 2 weeks ago by Brian Hoover
Related Packages: OpenG LabVIEW ZIP Library
   

Details

Thanks for making the update to help support a progress of the compression. But I think it isn't working correctly. If I set the block size to something that isn't -1, the offset seems wrong. I set it to 1000 and put it in a while loop until Finished is true, using a shift register to keep track of the File Transfer Info In and File Transfer Info Out. This shift register is initialized to 0 before entering the while loop.

Inside the Write File Offset is set to Length Rem from the ZLib Block Size. I don't think this is correct. Instead Offset should be the previous Offset, plus whatever the Block Length is from the ZLib Block Size. This way we will have Offset be 0 on the first iteration, then 1000 on the second iteration, then 2000 on the third iteration.

The File Refnum doesn't appear to be kept between calls either. If File Refnum in the File Transfer Info In is not a reference it will open it, but then it doesn't bundle it back into the File Transfer Info Out for the next iteration. I tried changing this but the next call the File Refnum is not a reference and it opens it again anyway.

Since the File Refnum keeps getting opened, it doesn't read the next N bytes. I think it keeps returning the first 1000 with every call, instead of the first 1000 then the next 1000. If I set the File Position to Offset before the read this works correctly but I don't think this should be necessary.

Comments

Rolf Kalbermatter Thanks for your report and yes you are right. The main problem is that I forgot to store the file refnum. The wiring for the offset is principally right but should be better called "remainder". I don't expect anyone going to mess with that refnum outside of this function so there should be no extra setting of the offset on the refnum necessary. I'm going to do this change and will create a new package later this week together with the fix for the password handling.
 •  Reply • 5 months
Rolf Kalbermatter But I would recommend to use a bit more than 1000 for a blocksize in your real code. Each loop iteration is going to take some time because of the API calls so there is a bit of a tradeoff between memory consumption and speed, but a buffer of 10000 or even 50000 bytes is nowadays peanuts.
 •  Reply • 5 months
Brian Hoover In regards to the blocksize. Yes of course it was just for testing where I wanted to see the file size grow and I picked a number I could see on a probe doing what I wanted. I'm unsure what block size I will actually pick but I'll probably do some testing. It looks like I used 4M as the block size before.
 •  Reply • 5 months
Rolf Kalbermatter Can you try again with the new release 5.0.3?
 •  Reply • 4 months
Brian Hoover Something still isn't right. The first thing I noticed is the Type input to the Store File Continue defaults to Invalid, instead of File like the control implies. This is pretty minor and I just put a constant of File on the block diagram. I still see the file refnum if opened in the Write File isn't bundled into the cluster. But more than that it seems like the File Open returns a "file refnum" that is always invalid according to the "Not a Number" primitive. I don't understand how the LVFile_OpenFile function works, but I don't understand why it always seems to return an invalid reference, when it clearly is valid. If I open the reference outside of the Store File Continue, and if I replace the "Not a Number" primitive in the Write File with a false constant, then it works as expected.
 •  Reply • 4 months
Rolf Kalbermatter After summer vacation got back to it and I see what I did there. . The quick solution is relatively easy and I have it on my system but there is actually a more systematic issue that I'm considering tackling here. Right now there is one more thing that came to my attention that I also am investigating. Might delay the next release a little to try to take that also into it.
 •  Reply • 2 months
Brian Hoover Hey that's fine if you want to wait. I'm most interested in Linux RT support myself. So while I'd be updating this code for other users of my stuff, I won't actually be going to 5.x myself until that support is added.
 •  Reply • 2 months
Rolf Kalbermatter Version 5.0.5 finally has a fully working chunked copy operation with ZLIB Store File First.vi and ZLIB Store File Next.vi
 •  Reply • 2 months

Please sign in to leave a comment.