1. Release Notes for Aridhia Data Selection Service

Release Notes for Aridhia Data Selection Service

Aridhia Data Selection Version:

1.1

Summary

The latest version of the Aridhia Dataset Selection Service primarily focusses on improving the stability of transferring datasets to Workspace while also improving backend elements, enhancing the user interface and fixing known minor bugs. Specifically this release includes:

  • Bug fixes to allow improve the reliability of transferring datasets to Workspaces, particularly large datasets (e.g. 14M-49M rows).
  • An enhancement to improve the reliability and timeliness of detecting and notifying the user of data transfer failures.
  • A bug fix to prevent redirection to http://app in multi-proxy environments or with IE11.
  • Query approval notification improvements where previously approved queries cannot be requested to be re-approved.
  • Query filtering enhancements where output columns that are then deleted, do not appear in a filter column dropdown list. Similarly, newly added output columns are added to filter column dropdown lists.

Released Features

Status Feature Description Reference

Bug Fix

Supporting large dataset transfers

Previously when transferring large datasets, the performance was either slower than expected or simply halted. This was due to the DST attempting to load the entire dataset into memory before transferring.

A configurable parameter has been added to set the number of rows to load from the source database before the DST begins to process and transfer data.

FAIR-181

Enhancement

Improving the reliability of data transfers

During the investigation of FAIR-181, further enhancements have been made to the data transfer process:

  • Unique cursors per job thread have been added to improve asynchronous job execution.
  • Improved job administration to avoid scenarios of multiple workers running the same job when multiple jobs are submitted in quick succession.

FAIR-333

Enhancement

Timeliness and reliability of transfer failure notification

  • A configurable parameter has been added to set the period to update job statuses.
  • Jobs are marked as failed in the event of unexpected service restarts
  • FAIR-181

    Bug Fix

    Preventing redirection to http://app

    In multi-proxy environments or when using IE11, sporadically the user would be directed to http://app. A bug fix has been released to avoid this.

    FAIR-327

    Bug Fix

    Do not allow request approval for previously approved request

    Previously when requesting approval for an already approved query, a dialog would appear signifying that approval has been requested.

    A bug fix has been released to prevent the user requesting approval for a previously approved query. The resulting dialog will now inform the user of this.

    FAIR-48

    Enhancement

    Clearly show query that approval was requested for

    It is now clear to see which queries have been sent for approval with 'Requesting Approval…' and 'Requested approval for transfer' labels.

    FAIR-330

    Enhancement

    Remove columns from filter dropdown when no longer present in Output Columns

    In the event a user removed columns from their query, the query filter dropdown list would still display this column. Such columns are now removed from the filter dropdown list.

    FAIR-328

    Enhancement

    Add columns to filter dropdown when added to Output Columns

    In the event a user added columns to their query, the query filter dropdown list would not display this new column. Such columns are now added to the filter dropdown list.

    FAIR-329

    1.0

    Summary

    The latest version of the Aridhia Dataset Selection Service primarily focuses on delivering improved dataset filtering capabilities, streamlined request then approve / deny processes and further user interface enhancements. Specifically, this release includes:

    • For logical 'in' and 'not in' filtering operations, delimited lists can now be given as input as opposed to individual filters per 'in' or 'not in' clause. 
    • Service users can now request for multiple queries to be approved or transferred to the Aridhia Workspace simultaneously, where Data Stewards can now concurrently approve or deny such approval requests. 
    • Improved user interface components from colour-coded query statuses, clearer wording of buttons and dialogs, to the addition of delete all buttons for filters and output columns. 

    Released Features

    Status Feature Description Reference
    Enhancement Adding list based 'in' / 'not in' filtering capabilities. The listed filtering operation now supports list arguments allowing users to filter lists based on multiple values (e.g. patient numbers). For example:
    • 'in' clause: only values matching those in the argument list are returned.
    • 'not in' clause: only values NOT matching those in the argument list are returned.
    The argument list supports the following delimiters: comma, newline, space, quotes, semicolon and can be used interchangeably within the same list. Furthermore, a de-identified field can be filtered using the original identifier values.
    XAP-10411
    Enhancement Sending multiple request transfer queries for approval concurrently Previously, a user had to individually select each query to request approval for transfer. Now users can select multiple (e.g. all) queries and concurrently request approval via the 'Request Approval' button. XAP-10412
    Enhancement Approving / denying multiple transfer requests concurrently Previously, a Data Steward had to individually select each query to approve or deny for subsequent transfer. Now Data Stewards can select multiple (e.g. all) queries and concurrently approve or deny via the 'Approve Request' and 'Deny Request' buttons respectively. XAP-10412
    Improvement Query actions should be present on the same pane as the query list The 'Actions' drop down list previously located on the top-right of the list of queries has been removed. Now query actions, such as clone, rename and delete are present on the query row itself, reducing the number of selects need to perform such actions. XAP10408
    Improvement Colour coded Query statuses It is now clearer to view the status of each query, which are now colour-coded and conform to Web Content Accessibility Guidelines (WCAG). XAP-10407
    Improvement Addition of per-table row numbers In order to gauge the size of a table, the number of rows per-table have been added to the right-hand side of each table name. Note: displayed row numbers are only updated when the service is restarted. Real-time reflection of row numbers will be resolved at a later date. XAP-10409
    Enhancement Delete all Filters button Previously in order to remove all filters from a query, each filter had to be removed individually via the 'Delete' button. A 'Delete All' button has been added to the top of the filter list which will clear the list of added filters. XAP-10406
    Enhancement Delete all Output Columns button Previously in order to remove all output columns from a query, each column had to be removed individually via the 'Delete' button. A 'Delete All' button has been added to the top of the output columns list which will clear the list of added filters. Note that output columns that are referenced by another query filter (e.g. 'in dataset') cannot be removed. A dialog has been added to list the affected output columns which cannot be removed via the 'Delete All' button. XAP-10410
    Improvement Reword 'Request Export' button and associated dialogs To conform to new language surrounding our Aridhia DRE, any references to 'export' (to workspace) has been changed to 'transfer'. XAP-10405