Longview Adapter - Source Metadata and Data Extraction
- *Willem Van Den Brink (Deactivated)
- Rachel Thomas
This page provides information on:
Performing a full extraction (metadata +data)
Performing a metadata only extraction for a given source
Performing a data extraction for the given source and year/period/scenario slice
Running Longview Data and Metadata Extraction
Running extractions via Command Line Interface (CLI)
To perform a full extraction (metadata + data) for the given source and year/scenario slice:
The command removes all existing data and metadata from fact database and imports data only for the years and scenarios specified in the arguments.
Command | full-extract |
|---|---|
Parameters |
|
Example |
|
To perform a metadata extraction for the given source:
The command removes all existing metadata from fact database and re-imports it again.
Command | metadata-extract |
|---|---|
Parameters |
|
Example |
|
To perform a data extraction for the given source and year/period/scenario slice:
The command removes all existing data from fact database and imports data only for the years, periods and scenarios specified in the arguments
Command | data-extract |
|---|---|
Parameters |
|
Example |
|
- For the view parameter, if you specify it as Periodic, the application only extracts the periodic data else extracts YTD. This parameter is optional, if not specified both Periodic and YTD periods will be extracted.
Running Extractions via Windows Scheduler
Pre-requisite:
Initial Source Metadata and Data Extraction has been completed by Professional Services.
Configuring Data Loads
To reconfigure the dimension members for the data load, the related. Lvparam file needs to be edited. After saving the file (do not change the name of the file), windows scheduler will automatically use the updated configuration.
Parameter files (.lvparam files)
Each job comes with it’s own parameter file from which it reads parameter values and extracts metadata and data based on it. By default, there are three such files:
Script | Parameter file | Parameters |
|---|---|---|
MetadataExtraction.ps1 | MetadataExtractionParameters.lvparam | SOURCE_NAME |
DataExtraction.ps1 | DataExtractionParameters.lvparam | SOURCE_NAME, YEARS, SCENARIOS, PERIODS (optional) |
FullExtraction.ps | FullExtractionParameters.lvparam | SOURCE_NAME, YEARS, SCENARIOS |
Parameters:
SOURCE_NAME - Name of the source from which metadata or data is extracted
YEARS - One or more years that are comma separated and specify years for which data is extracted
SCENARIOS - One or more scenarios that are comma separated and specifies which scenarios are extracted
PERIODS - Only applicable for DataExtraction.ps1 script which specifies periods which want to be extracted for the years, can be Q1-Q1 or months from 1-12
To run a job, its parameter file needs to have the appropriate configuration using comma separated values.
Example with periods in the file:
If extraction isn’t needed to be executed with periods file can be edited like this:
Scheduled extraction
After completing the process above, scheduled jobs can be run automatically using Windows Scheduler. More information about this tool can be found here.
Manual extraction
Besides the scheduled runs, it is possible to run manual extractions. They can also be triggered in Windows Scheduler:
After selecting the desired job, you can ‘run’ the item in the right bottom panel of the screen.