Page 1 of 1
Can't read unicode files, totally stuck at the moment
Posted: Thu Sep 26, 2013 12:33 am
by deploy
I am trying to parse a log file generated from an external (windows) command my setup need to execute.
It does not work, read from text file does not handle unicode when reading:
Read from Text File #READALL\c:\Windows\security\logs\scesrv.log into Variable sec_log
I have seen some posts about bugs in unicode handling and one suggestion was to add an empty line at the start of the file, but if I try:
Write into Text File c:\temp\scesrv.log from Value RESULTS: (at start of file)
IA will delete the file contents if the file is unicode!
At the moment I am totally stuck in my project due to this.
I am on IA 17 build 8.6.13 where "everything" should be unicode compatible from what I understand.
Please advice?!
Re: Can't read unicode files, totally stuck at the moment
Posted: Thu Sep 26, 2013 4:06 pm
by FrancescoT
Dear ,
are you totally sure that your file is UNICODE?
For what I know, The file content should be deleted with trying to add an empty string if it is an ANSI file ... and not UNICODE.
Regards
Re: Can't read unicode files, totally stuck at the moment
Posted: Thu Sep 26, 2013 10:55 pm
by deploy
Dear FrancescoT,
I am assuming unicode since there is a BOM of FFFE and each character is 16 bytes:

- uni_hex.PNG (7.44 KiB) Viewed 11200 times
When read by IA the results are (in the messagebox):

- uni_out.PNG (19.5 KiB) Viewed 11200 times
When I try to append an empty line as a workaround the contents of the file is deleted.
I am attaching a file of this type (scesrv.log)
Regards,
Re: Can't read unicode files, totally stuck at the moment
Posted: Mon Sep 30, 2013 8:58 am
by FrancescoT
Dear Deploy,
just tried ... use UTF-8 or UTF-8 (without BOM) files instead.
Regards
Re: Can't read unicode files, totally stuck at the moment
Posted: Tue Oct 01, 2013 12:31 am
by deploy
FrancescoT wrote:Dear Deploy,
just tried ... use UTF-8 or UTF-8 (without BOM) files instead.
Regards
Dear FrancescoT,
Did not work in your test either I take it?
The output file is from secedit.exe created at runtime when my installer applies security policies as part of the software installation. Are you suggesting that I from within IA somehow could convert the file from original format to something that IA can read? If I had control over the output format to begin with I would not be totally stuck on this
Regards
Re: Can't read unicode files, totally stuck at the moment
Posted: Tue Oct 01, 2013 10:47 am
by FrancescoT
Dear Deploy,
I am just saying that IA 16 or 17 can read an ANSI text file or an Unicode UTF-8 (with or without BOM encoding) text file.
Regards
Re: Can't read unicode files, totally stuck at the moment
Posted: Tue Oct 01, 2013 11:26 pm
by deploy
Dear FrancescoT,
Just to clarify, is this a bug or is it just that IA will not support "the Microsoft standard" of encoding unicode text in UTF-16? (I get the impression MS uses FFFE, see
http://msdn.microsoft.com/en-us/library ... 85%29.aspx)
I am asking because I need to know if I have to write my own code to parse unicode files for results or if I can expect IA to do so in a near future?
Regards
Re: Can't read unicode files, totally stuck at the moment
Posted: Wed Oct 02, 2013 11:20 am
by FrancescoT
Dear Deploy,
It is not a bug, but as I said previously, currently only those formats are supported.
I believe that the other formats will be supported in future ... but at the moment, I'm not able to confirm any time frame.
Regards
Re: Can't read unicode files, totally stuck at the moment
Posted: Thu Nov 21, 2013 12:44 am
by deploy
If there's anyone else stuck due to not being able to read UTF-16 files (typically output from most MS console applications and cmd), here's a quick and dirty solution:
Use "type" to convert to ansi!It isn't pretty but it works... For example here's how to get the bios serial number. It will be obsolete of course once IA can handle UTF-16 files.
Code: Select all
Run Program wmic.exe /output:c:\tmp\ser.txt bios get serialnumber (WAIT)
Run Program cmd.exe /c type c:\tmp\ser.txt >c:\tmp\serial.txt (WAIT)
Read from Text File c:\tmp\serial.txt into Variable ComputerSerial
Read from Text File c:\tmp\serial.txt into Variable ComputerSerial
Parse String $ComputerSerial$ into Variables ComputerSerial and (Split at first occurrence of pattern)
Convert Variable ComputerSerial to UPPER case
Re: Can't read unicode files, totally stuck at the moment
Posted: Thu Nov 21, 2013 11:31 am
by FrancescoT
thank you for sharing it!