Documentation Index
Fetch the complete documentation index at: https://docs.scanoss.com/llms.txt
Use this file to discover all available pages before exploring further.
The kb-download.sh script provides a unified interface for retrieving the full Knowledge Base (KB), a KB update, or the test KB directly from the SCANOSS SFTP server. It connects to the server, displays the available versions and allows you to select the desired package for download. Before proceeding, it verifies that sufficient disk space is available.
For KB updates, the downloaded package includes a separate import utility (ldb-import.sh) which is used to import the update into your local LDB instance.
KB Download Prerequisites
- lftp (recommended): Enables faster, parallel, and resumable downloads. You can install it using
apt install lftp. If lftp is not available, the script can fall back to the standard sftp client. When running with the -y flag, the fallback to sftp is handeld automatically.
- sshpass (required only for sftp fallback): Needed to provide the password non-interactively when using
sftp. Install it with apt install sshpass. This dependency is not required when using lftp.
- SFTP credentials: Supplied by the SCANOSS sales team, including host, port, username and password.
Running the Script
Make sure the script has execution permissions:
chmod +x install/kb-download.sh
Command-line Options
All connection details, paths, and the download mode can be passed as arguments to skip the matching interactive prompt:
./kb-download.sh -m update -h sftp.test.xyz -P 12345 -u USER21 -p mypassword
| Flag | Description |
|---|
-m | Download mode: full (full KB), update (KB update), or test (test KB) |
-h | SFTP host |
-P | SFTP port |
-u | SFTP username |
-p | SFTP password |
-t | lftp parallel download threads (default: 25) |
-d | Force download tool: lftp or sftp |
| Flag | Description |
|---|
-V | KB version (full/update mode); default: latest from LATEST.txt on the server |
-o | Destination path. Test mode: test-KB destination. Full mode: oss folder destination. (default: /var/lib/ldb/oss) |
-r | Full mode only: destination for non-oss items (default: /tmp/scanoss_kb_full_<version>) |
-D | Update mode only: download base directory (default: /tmp/scanoss_kb_update; final path is <D>/<version>) |
| Flag | Description |
|---|
-y | Don’t prompt; use defaults for unspecified values and auto-confirm the download. Required values (-m, -h, -P, -u, -p) must be supplied — the script exits with an error if any are missing. |
-f | Force download even if free disk space is below the required size. Only meaningful with -y; without -y you are prompted to confirm. |
-? | Show help |
-y is set.
Non-interactive Examples
# Defaults (latest version, /var/lib/ldb/oss + /tmp/scanoss_kb_full_<version>):
./kb-download.sh -y -m full -h sftp.test.xyz -P 12345 -u USER21 -p mypassword
# Pin a version, custom paths, force on low disk space:
./kb-download.sh -y -f -m full -V 26.03 \
-o /data/oss -r /data/kb-rest \
-h sftp.test.xyz -P 12345 -u USER21 -p mypassword
# Update mode with a custom download base directory:
./kb-download.sh -y -m update -D /opt/scanoss/updates \
-h sftp.test.xyz -P 12345 -u USER21 -p mypassword
# Test KB:
./kb-download.sh -y -m test -h sftp.test.xyz -P 12345 -u USER21 -p mypassword
How It Works
- Connection: the script prompts for (or reads from arguments) the SFTP host, port, username, and password
- Mode selection: you choose between downloading the full KB, a KB update, or the test KB
- Version discovery (full/update only): it connects to the SFTP server and lists all available versions of the chosen type, marking which one is the latest. Test KB has no versions — there is only one current test KB on the server.
- Selection (full/update only): you choose which version to download (defaults to the latest)
- Disk space check: the script fetches the metadata and verifies there is enough free disk space before downloading. If not enough is available, you are warned and asked to confirm before continuing.
- Download:
- For full KB: the download is split between two destinations. The
oss folder (the main LDB data) goes to its own destination (default: /var/lib/ldb/oss), and all remaining files and folders go to a separate directory (default: /tmp/scanoss_kb_full_<version>). Both destinations can be changed when prompted.
- For updates: the whole update folder is downloaded to a single directory (default:
/tmp/scanoss_kb_update/<version>).
- For test KB: the test KB’s
oss folder is downloaded to a single destination (default: /var/lib/ldb/oss). This will populate your LDB with the test data for verification.
- Import (updates only): after downloading an update, run the
ldb-import.sh script included in the downloaded folder
Example Sessions
Update
SCANOSS Knowledge Base Download
================================
Using lftp for downloads (parallel, resumable).
What do you want to download?
1) Full KB
2) KB update
3) Test KB
Select [1-3]: 2
SFTP host: sftp.test.xyz
SFTP port: 12345
SFTP username: USER21
SFTP password:
Fetching available update versions...
Available update versions:
-------------------
1) 26.01
2) 26.02
3) 26.03 (latest)
-------------------
Select a update version to download [1-3] (default: 3):
Selected update: 26.03
Download directory [/tmp/scanoss_kb_update]:
Fetching update metadata...
Update size: 5.2G
Free space: 120G (on /dev/sda1)
Disk space OK.
lftp parallel threads [25]:
Download update 26.03 to /tmp/scanoss_kb_update? [Y/n]
Downloading with lftp (25 parallel threads, resumable)...
Update downloaded to /tmp/scanoss_kb_update/26.03
Finished downloading update.
Please run the ldb-import.sh script provided inside the update folder to import into the KB:
/tmp/scanoss_kb_update/26.03/ldb-import.sh
Full KB
./kb-download.sh -m full -h sftp.test.xyz -P 12345 -u USER21
SCANOSS Knowledge Base Download
================================
Using lftp for downloads (parallel, resumable).
SFTP password:
Fetching available full KB versions...
Available full KB versions:
-------------------
1) 26.01
2) 26.02
3) 26.03 (latest)
-------------------
Select a full KB version to download [1-3] (default: 3):
Selected full KB: 26.03
Destination for 'oss' folder [/var/lib/ldb/oss]:
Destination for remaining files/folders [/tmp/scanoss_kb_full_26.03]:
Fetching full KB metadata...
Full KB size: 210G
Free space: 500G (on /dev/sda1)
Disk space OK.
lftp parallel threads [25]:
Download full KB 26.03 to /var/lib/ldb/oss + /tmp/scanoss_kb_full_26.03? [Y/n]
Downloading kb/full/26.03/oss with lftp (25 parallel threads, resumable)...
Downloading kb/full/26.03/md5sum with lftp (25 parallel threads, resumable)...
Downloading kb/full/26.03/26.03_files.md5sum with lftp (25 parallel threads, resumable)...
Full KB downloaded:
oss folder: /var/lib/ldb/oss
other files: /tmp/scanoss_kb_full_26.03
Finished downloading full KB.
Test KB
./kb-download.sh -m test -h sftp.test.xyz -P 12345 -u USER21
SCANOSS Knowledge Base Download
================================
Using lftp for downloads (parallel, resumable).
SFTP password:
Destination for test KB [/var/lib/ldb/oss]:
Fetching test KB metadata...
Test KB size: 180M
Free space: 500G (on /dev/sda1)
Disk space OK.
lftp parallel threads [25]:
Download test KB to /var/lib/ldb/oss? [Y/n]
Downloading kb/test/oss with lftp (25 parallel threads, resumable)...
Test KB downloaded to /var/lib/ldb/oss
Finished downloading test KB.
Verifying a KB Update
After importing an update, verify the installation by scanning the test WFP files included in the update directory:
scanoss-py scan /tmp/scanoss_kb_update/26.03/26.03_test_file.wfp
scanoss-py scan /tmp/scanoss_kb_update/26.03/26.03_test_snippet.wfp
If the import is interrupted, it may cause corruption in the existing LDB.
Next Steps
Once the Knowledge Base installation is complete, proceed to Verify Installation.
