Can't read unicode files, totally stuck at the moment

[deploy]

I am trying to parse a log file generated from an external (windows) command my setup need to execute.

It does not work, read from text file does not handle unicode when reading:
Read from Text File #READALL\c:\Windows\security\logs\scesrv.log into Variable sec_log

I have seen some posts about bugs in unicode handling and one suggestion was to add an empty line at the start of the file, but if I try:
Write into Text File c:\temp\scesrv.log from Value RESULTS: (at start of file)
IA will delete the file contents if the file is unicode!

At the moment I am totally stuck in my project due to this.

I am on IA 17 build 8.6.13 where "everything" should be unicode compatible from what I understand.
Please advice?!

[FrancescoT]

Dear ,

are you totally sure that your file is UNICODE?
For what I know, The file content should be deleted with trying to add an empty string if it is an ANSI file ... and not UNICODE.

Regards

[deploy]

Dear FrancescoT,

I am assuming unicode since there is a BOM of FFFE and each character is 16 bytes:

uni_hex.PNG (7.44 KiB) Viewed 10921 times

When read by IA the results are (in the messagebox):

uni_out.PNG (19.5 KiB) Viewed 10921 times

When I try to append an empty line as a workaround the contents of the file is deleted.

I am attaching a file of this type (scesrv.log)

Regards,

[FrancescoT]

Dear Deploy,

just tried ... use UTF-8 or UTF-8 (without BOM) files instead.

Regards

[deploy]

Dear Deploy,
just tried ... use UTF-8 or UTF-8 (without BOM) files instead.
Regards

Dear FrancescoT,

Did not work in your test either I take it?

The output file is from secedit.exe created at runtime when my installer applies security policies as part of the software installation. Are you suggesting that I from within IA somehow could convert the file from original format to something that IA can read? If I had control over the output format to begin with I would not be totally stuck on this

Regards

[FrancescoT]

Dear Deploy,

I am just saying that IA 16 or 17 can read an ANSI text file or an Unicode UTF-8 (with or without BOM encoding) text file.

Regards

[deploy]

Dear FrancescoT,

Just to clarify, is this a bug or is it just that IA will not support "the Microsoft standard" of encoding unicode text in UTF-16? (I get the impression MS uses FFFE, see http://msdn.microsoft.com/en-us/library ... 85%29.aspx)

I am asking because I need to know if I have to write my own code to parse unicode files for results or if I can expect IA to do so in a near future?

Regards

[FrancescoT]

Dear Deploy,

It is not a bug, but as I said previously, currently only those formats are supported.

I believe that the other formats will be supported in future ... but at the moment, I'm not able to confirm any time frame.

Regards

[deploy]

If there's anyone else stuck due to not being able to read UTF-16 files (typically output from most MS console applications and cmd), here's a quick and dirty solution:
Use "type" to convert to ansi!

It isn't pretty but it works... For example here's how to get the bios serial number. It will be obsolete of course once IA can handle UTF-16 files.

Code: Select all

 Run Program wmic.exe /output:c:\tmp\ser.txt bios get serialnumber (WAIT)
  Run Program cmd.exe /c type c:\tmp\ser.txt >c:\tmp\serial.txt (WAIT)
  Read from Text File c:\tmp\serial.txt into Variable ComputerSerial
  Read from Text File c:\tmp\serial.txt into Variable ComputerSerial
  Parse String $ComputerSerial$ into Variables ComputerSerial and  (Split at first occurrence of pattern)
  Convert Variable ComputerSerial to UPPER case
 

[FrancescoT]

thank you for sharing it!