Your Knoema Enterprise subscription may include a specific level of support for data integration and maintenance as well as customized small group training. Contact your Knoema representative to learn more.
Depending on your role, you will have different options with regard to dataset updates:
- System admins (managers) may update datasets uploaded directly to the platform. All data sourced from Knoema.com and 3rd party sources managed by Knoema can be updated by Knoema only.
- Non-admin users can only update private datasets (if your system allows them to upload data) or those datasets that other users have granted a specific user editorial permissions. These users will have a special data ‘role’ enabled by the system admin to verify the datasets as well. Otherwise, an admin will need to verify the datasets and the user will see a warning during the upload that the dataset requires verification.
There are four ways a dataset can be updated:
1 —Add/remove values.
Use it when you need to upload newly available data or to modify prior data
Example: Upload new GDP data for 2020
2 — Add element(s) to dimension(s).
Use it when you need to add a new element (such as a location or a variable) to a dimension
Example: A new indicator appeared in a report
3 — Overwrite the whole dataset.
Use it when (1) you have an initial data file and want to update a large number of new/revised time series, (2) you need to remove any element from a dimension
4 — Replace dataset.
Use it when you need tomake structural changes to your dataset (e.g. add/remove a dimension)
Add/remove values
To add new data in an existing dataset, start by downloading the initial data and dataset structure:
- Download the portion of the dataset that you need to update by first selecting the data of interest in the Dataset Viewer (locations, indicators, measures etc).
- Click the Download data only button in the right panel.
- Check the registered e-mail address associated with your account. You will receive the file with the data worksheet.
- Now, let’s imagine that you need to add data for the US population, 2020. In the file, add a new column with 2020 population data.
Warning! Upload the 'Data' sheet only; if you upload the whole data file with changed structure, you'll overwrite THE WHOLE dataset.
- Next, go back to the dataset in the dataset viewer and click More Actions > Upload data and upload the file (only the 'Data' sheet) you prepared. If your update was correct, you will see an upload report that summaries the number of values you’ve added.
Note: In the Update Notes field a manager could describe the data, methodology, or other information pertaining to the data update. If the dataset is public, users must resubmit the datasets for verification after each update.
Note: If you have received an ‘Unexpected file format’ error or experienced a technical issue, contact your Knoema representative for assistance.
To remove data from an existing dataset, you’ll need to follow the same steps as those used for adding data. The only difference is that instead of inserting new columns or values into your data file, you will be removing them using empty cells or columns in the data file.
Here’s a snapshot of an Excel file showing empty columns that we can use to remove data for the years 2000-2004:
Your upload report will show the number of Deleted data points, as shown below:
Note: In the Update Notes field a manager could describe the data, methodology, or other information pertaining to the data update. If the dataset is public, users must resubmit the datasets for verification after each update.
Note: If you have received an ‘Unexpected file format’ error or experienced a technical issue, contact your Knoema representative for assistance.
Add element(s) to dimension(s)
You can add new elements to existing dataset dimensions. To do so, start by downloading the complete dataset file:
- Open your dataset in the Dataset Viewer and click More Actions > Upload Data.
- Click the link for Request a complete dataset. The file will be delivered to the e-mail address associated with your account.
- Since you are using a regular dataset structure, add new elements to the dimension sheet with code. In the image below, the user is adding a new measure and thus is in the Measure dimension worksheet.
- To avoid removing all of the data from your dataset, go to the Dataset sheet and remove all rows except Dataset Name and the Dimensions list.
- Now upload your new file to the dataset from the Dataset Viewer (More Actions > Upload data).
- To upload data for the new element(s), repeat steps from Add/remove values.
Note: In the Update Notes field a manager could describe the data, methodology, or other information pertaining to the data update. If the dataset is public, users must resubmit the datasets for verification after each update.
Note: If you have received an ‘Unexpected file format’ error or experienced a technical issue, contact your Knoema representative for assistance.
Warning! If your dataset exceeds 10MB, you will not be able to download the whole dataset with data, but you may request the complete dataset structure.
Overwriting the whole dataset
Overwriting means you are deleting all the data in the existing dataset and replacing it with data from a fresh file. No structural changes are made during this process.To overwrite all data in a dataset:
- Open your dataset in the Dataset Viewer and click More Actions > Upload Data.
- Click the link for Request a complete dataset.* The file will be delivered to the e-mail address associated with your account.
- Within the file you received by e-mail, make all of your data changes within the worksheet named Data (your name for this worksheet may vary; the worksheet you are editing contains all the data values).
- From the dataset viewer > More Actions > Upload data, reupload your revised file.
Note: In the Update Notes field a manager could include information about the data, methodology, or other pertaining to the update. Users must resubmit the datasets for verification after each update if the dataset is public.
Note: If you receive an ‘Unexpected file format’ error or experience any other technical issue, contact your Knoema representative for assistance.
Warning! If your dataset exceeds 10MB, you will not be able to download the whole dataset with data, but you may request the complete dataset structure.
Replace dataset
In some instances you may need to make structural changes to your dataset. In this scenario, you must upload the new dataset, as previously explained, and set it as a ‘replacement’ dataset for the original version as follows:
- To simplify the process, start by bookmarking the new dataset that you have uploaded.
- Now, return to the data catalog to find and open the previous version of the dataset that you want to replace.
- In the Dataset Viewer, select the menu More Actions > Edit Metadata.
- Scroll down to the Replacement Dataset and select None.
- Select the dataset you've added to Bookmarks in step 1 and click OK.
- The replacement dataset field now shows the title of the replacement dataset. Select Save.
- The older version of the dataset now shows a yellow warning banner in the data browser—as well as in any visualization/dashboard built on the dataset—to alert users that a newer version of the dataset is available and provides a link to the newer version.