Including Quality Codes for Values in KiDAT Tasks

[SteveWKISTERS]

A few questions have arisen regarding including a Quality code field within the data scraped by KIDAT Tasks. The following are the steps required in order to modify your KiDAT tasks to do so.

Note that if you'd like to include Remarks within your KiDAT Task, you simply need to map the Remarks column location within the task.


1. Edit your KIDAT Task and open the Advanced Options tool in the top left.

2. Select Modify the ZXRP Header and modify your Layout string to include the status field before the remark (as depicted in the string below) and click OK.

LAYOUT(timestamp,value,status,remark)

3. Open the Advanced Options tool in the top left and select Replace Invalid Values. When the next dialog prompts you to Enable this feature, select No


You're KIDAT task will now accept WISKI Quality Codes in the "Status" column as organized in your layout. You can use a variety of methods to capture this information (i.e. defining the column value, etc.). These will be made available through the KiScript Library available on the GRCA's secure BizPortal site.

https://bizportal.grandriver.ca

[cmcguire]

Thanks Steve,

We have a python script that converts the HYDAT CSV data to .ZRXPs.  Didn't realize it was this easy to modify the script to include the WSC quality codes.

Does anyone have a KiDAT task that processes the HYDAT CSVs?

Chris

[cmcguire]

Hi Steve,

In regards to the ZRXP files, what should go in the header layout if I wanted to also include data for the "Tags" column?

T Daniel Dickson

[SteveWKISTERS]

Hi Chris,

Can you send me one of your HYDAT CSV's? There might be one floating around somewhere but I won't know until I test out some likely candidates.


@ Chris / Daniel RE: Including Tags in KIDAT Tasks

Yes, Tags can also be included in KiDAT tasks, but the approach is determined by the type of tag - numeric or string.

As a disclaimer, you should have updated your KiDAT system to version 2.7.5.8 to do this. This is also possible with version 5.6 but I updated before I tried it so I can't confirm 100%. This import also assumes you are importing quality codes, tags, and remarks all in one file.

If you aren't importing Quality, Tag, and Remark all together then you have the option of simply subbing one of the fields that aren't in use to be the Tag field.



Steps:

1. Similar to the instructions above, you will need to update your Header to include your field. In this case, you want to modify it to include "timestamp,value,quality,tag,remark". Note that the tag field must fall between quality and remark.

2. In your KiDAT task, we need to essentially define two locations within the "Quality Code Location" field, separated by a space. The first is the actual column that contains quality, the second is for the tag field.

3. For numeric tag values, all you need to do is define a the column as you would for any other KiDAT field. Listed below is an example of the input, where the first Cell statement refers to Quality and the second refers to Tag.

Cell(0,<CurrentRow>) Cell(1,<CurrentRow>)

4. For string values, you need to turn on the Extended Functionality. Note: this will require that you redefine other areas of your task to conform to the Extended Functionality rules, so you may wish to make a "Test" task first.

A. Use the same statement as listed above, but include the separation '&' and quote 'chr(34)' characters to define the string boundaries. See example below - again first Cell statement refers to Quality code and second Cell Statement (beginning at "& chr(34)) refers to Tag value.

Cell(0,<CurrentRow>) & chr(34) & Cell(1,<CurrentRow>) & chr(34)


Steve with Kisters

[cmcguire]

Hi Steve,

I'm in the process of formatting zrxp files for our staff gauges and I noticed that the remarks column contained spaces.  As this also appears to be the value separator for the zrxp files and as I assume spaces in remarks will be fairly common, I was wondering how to include these values.  Should treat it like a string and surround it with quotes?

Thanks,

T Daniel Dickson


EDIT:

After some experimentation and reading, it seems that WISKI stores both tags and remarks in one column.  Is it possible, then, to import them both through ZRXPs?  I should also mention that we are (currently) creating the ZRXPs from scratch with a python script, and so it would be useful to know if this is possible without hydat.

[SteveWKISTERS]

Hi T Daniel Dickson,

Have you experimented with creating these files? I've used the approach discussed above to import a remark with multiple spaces (i.e. "Imported from Source 1") without issue. I believe that once you've identified the column as a remark KiDAT should be able to handle it. That said, if this is causing you problems then surrounding your remarks in quotes should solve your problem by better delineating your strings.


For clarity, I'm fairly certain that while WISKI displays both tags and remarks in one column - they are not stored in the same column. They can both be imported in through one file. Have you tried the approach outlined above regarding modifying the expected inputs to include tags and remarks?

Are you looking to move from the python script to KiDAT? I'm confused as to why you would develop two processes to achieve this. If you have an existing process that you're looking to replicate it may be best to schedule a meeting with Support to evaluate this more fully.

Steve with KISTERS 

[cmcguire]

Hi Steve,

I've been testing KiDAT to see if I can create a ZRXP with status, tags, and remarks.  I can get both status and tags together; however, when I try to put 2 columns into the quality column as you suggest, it ignores both and only outputs remarks.  I've also tried manually importing a ZRXP I created and WISKI doesn't recognize the "tag" column; I mention this because I've noticed that the header that gets exported by KiDAT is exactly the same as the one I enter, so even if I were able to get it to export both status and tags, I expect that it would not import correctly.

T Daniel Dickson

[SteveWKISTERS]

Hi Daniel,

I spoke with Brandon about this issue this morning and showed him the task that we had created to handle this. It turns out that there is a check in place to remove any non-numeric Quality values from going into KiDAT. Brandon has adjusted this in the code and released a new version that will accept the data we're looking to import. Please update your KiDAT and give it a shot - you may need me to activate it again - if that's the case, please contact me and I'll get a Teamviewer scheduled with either yourself or Chris.

Cheers,

Steve with KISTERS