Merge branch 'updatedocs' into 'master'

Extend tracker documentation

See merge request security-tracker-team/security-tracker!101

1 files changed, 148 insertions, 6 deletions

diff --git a/doc/security-team.d.o/security_tracker b/doc/security-team.d.o/security_tracker
index 7a42cbb61a..74d3687271 100644
--- a/doc/security-team.d.o/security_tracker
+++ b/doc/security-team.d.o/security_tracker
@@ -16,6 +16,18 @@ online. Everything is designed to be very simple to use, transparent and
 easy to see what other people are working on so you can work on other
 things.
 
+The Debian Security Tracker is only concerned with how specific vulnerabilities affect
+Debian. Many vulnerabilities are triaged as NFU (`NOT-FOR-US`) simply because the
+vulnerable software is not (yet) packaged for Debian. Triage comments on any specific
+vulnerability only reflect the possible impact on a system running Debian.
+
+For example, systems with some additional or modified packages compared to Debian need
+a separate triage process for every NFU to find ones which are relevant to what has
+been added as well as a triage on packages which differ from Debian.
+
+Entries in the Debian Security Tracker do not imply anything about how a vulnerability
+may affect systems other than Debian.
+
 Gentle Introduction
 -------------------
 
@@ -421,6 +433,11 @@ assess these levels.
 Certain packages may get higher or lower rating than usual, based on
 their importance.
 
+Assessments of severity are made against the binaries as provided by Debian. For each
+vulnerability, the severity assigned within the Debian Security Tracker only relates to
+how Debian views that vulnerability and how quickly the fix may need to be applied to
+the specified package(s) within Debian.
+
 ### Vulnerabilities without an assigned CVE id
 
 If you learn of a vulnerability to which no CVE id has been assigned yet, you can
@@ -541,22 +558,65 @@ cross-reference will be added automatically by the cron job. However,
 you do need to add `[lenny]` or `[squeeze]` entries to `CVE/list` when there
 is a `no-dsa` or `not-affected` condition.
 
+Summary of tracker syntax
+-------------------------
+
+For a vulnerability in a package in Debian or proposed for introduction into Debian,
+the syntax should contain at least the `PKG_NAME` tabbed line and a `NOTE:` providing a
+URL to useful references, like commit references, bug tracker entries and advisories.
+Other lines are added, where relevant, within the general syntax.
+
+    CVE-YYYY-NNNNNN [(description)]
+     \t RESERVED
+     \t - PKG_NAME [PKG_TAG | PKG_FIX_VERSION] SEVERITY_LEVEL (free text comment)
+     \t [codename] - PKG_NAME [PKG_TAG | PKG_FIX_VERSION] (free text comment)
+     \t NOTE:
+     \t TODO:
+
+- Each tabbed line, except `RESERVED`, can be repeated, e.g. for code embedded in
+  multiple packages and/or to cover multiple suites. Codenames are listed in order of
+  the release date.
+- PKG_NAME is the source package name in the archive.
+- PKG_TAG : `<no-dsa>` | `<unfixed>` | `<undetermined>` | `<not-affected>` | `<itp>`
+- SEVERITY_LEVEL : `(unimportant)` | `(low)` | `(medium)` | `(high)`
+- The pre-commit hook will check the syntax of each entry.
+
+The description of the CVE is not edited in the security tracker but it will be
+shortened in the tracker page for the vulnerability. A temporary description can be
+added with the `[description]` syntax, for example for clarification. This will not be
+overridden by an automatic update unless there is a change in the description of the
+CVE in the MITRE feed.
+
+For `<itp>`, the comment needs to include the bug number as `(bug #NNNNNNNNNN)`. (The
+`<itp>` package tag is used for both ITP and RFP bugs -
+see [ITP/RFP packages](#issues-in-itp-andor-rfp-packages))
+
+`NOTE:` annotations are often used for URLs for more information but can also be
+used for descriptive comments.
+
 Checking in your changes
 ------------------------
 
 After thoroughly researching each issue (as described above) and editing
 the relevant files, commit your changes. Peer review is (hopefully) done via the
 mailing list and IRC notifications (see [Automatic issue updates](#automatic-issue-updates) above).
-However, changes to the tracker website itself (e.g., the files in lib/*
-and bin/tracker_service.py) should be vetted and approved before being
+However, changes to the tracker website itself (e.g., the files in `lib/*`
+and `bin/tracker_service.py`) should be vetted and approved before being
 committed. The preferred way to do this is to send a patch to the
-debian-security-tracker@lists.debian.org mailing list.
+`debian-security-tracker@lists.debian.org` mailing list or a merge request in Salsa.
+
+- [Salsa](https://salsa.debian.org/security-tracker-team/security-tracker/)
+- [https://lists.debian.org/debian-security-tracker/](https://lists.debian.org/debian-security-tracker/)
 
 Commits are checked for syntax errors before they are actually committed,
 and you'll receive an error and your commit is aborted if it is in error.
 To check your changes yourself beforehand, use `make check-syntax` from
 the root of the Git directory.
 
+Note: It can be useful to use `git worktree` support for merging changes to master and
+ease issues that can occur when someone else has committed in between. See [git
+worktree (1)](https://manpages.debian.org/unstable/git-man/git-worktree.1.en.html).
+
 Following up on security issues
 -------------------------------
 
@@ -573,7 +633,7 @@ Tracking of security bugs in the BTS and linking them to a user tag by CVE
 --------------------------------------------------------------------------
 
 There's an automated tagging of security-related bugs to CVE IDs through
-the user tag security for the user debian-security@lists.debian.org.
+the user tag security for the user `debian-security@lists.debian.org`.
 
 All bugs added to the tracker are automatically tagged. You can use
 the search
@@ -594,11 +654,93 @@ with the following content:
 Contributing with the security tracker code
 -------------------------------------------
 
-Either fill a bug against the security-tracker pseudo-package attaching the patch
-to be reviewed or create a merge request for the security-tracker project.
+Either file a bug against the `security-tracker` pseudo-package attaching the patch
+to be reviewed or create a merge request for the security-tracker project in Salsa.
+
+### Helper scripts for one-off updates
+
+On success, scripts output a snippet of the main CVE list showing the new CVE
+information. Make sure to check for warnings and errors reported by the script. The
+output file needs to be manually reviewed and can then be merged using
+`./bin/merge-cve-files` or sent for review by the security team by email.
+
+##### Updating a vulnerability
+
+* Mark a given released suite as not affected for a specific CVE and source package:
+
+    `./bin/update-vuln --cve CVE --src SRC --suite SUITE`
+
+* Add a bug number to an existing CVE entry
+
+    `./bin/update-vuln --cve CVE --number 1000000`
+
+* Add a note to a specific CVE entry
+
+    `./bin/update-vuln --cve CVE --note "quoted note string"`
+
+Example workflow:
+
+    ./bin/update-vuln --cve CVE-YYYY-NNNNN ...
+
+check for error and warning messages & merge into the main CVE list:
+
+    ./bin/merge-cve-files ./CVE-YYYY-NNNNN.list
+
+review change to data/CVE/list
+
+    git diff data/CVE/list
+    rm ./CVE-YYYY-NNNNN.list
+
+.. repeat for additional entries to this or other CVEs.
+
+    git add data/CVE/list
+    git commit
+
+#### Retrieve fixes in uploads to unstable
+
+`./bin/grab-cve-in-fix` supports different ways to retrieve one or more CVEs as fixed in unstable:
+
+- Using information directly from the upload into unstable:
+
+    `cat changes | ./bin/grab-cve-in-fix --input`
+
+- Using information in the lists.debian.org archive:
+
+    `./bin/grab-cve-in-fix --archive https://lists.debian.org/debian-devel-changes/2021/12/msg01280.html`
+
+- Using information in the package tracker:
+
+    `./bin/grab-cve-in-fix --tracker https://tracker.debian.org/news/1285227/accepted-freerdp2-241dfsg1-1-source-into-unstable/`
+
+- Using local caches in the security-tracker:
+
+    `./bin/grab-cve-in-fix --src SRC --cves [CVES...]`
+
+Note: to use `STDIN` with the --input option, the changes content must be signed - i.e.
+as it would appear in notifications after the upload. This can be used to double-check
+your CVE list before uploading to ftp-master. `./bin/grab-cve-in-fix` will report if a
+CVE does not exist or if the CVE is attributed to a different package.
 
 **TODO** (further details)
 
+### Contributing ongoing triage work
+
+Some familiarity with the tooling and syntax will be needed for this, as with any development
+project.
+
+* `./bin/check-new-issues` - use the -h option to see the help output.
+
+* `./bin/report-vuln` - generate the correct email body to report a bug against a source package
+  relating to an unfixed CVE(s).
+
+### Useful search support for checking new CVEs
+
+- [https://www.debian.org/distrib/packages#search_packages](https://www.debian.org/distrib/packages#search_packages)
+- [https://wnpp.debian.net/](https://wnpp.debian.net/) (Be aware, forwarded ITPs might
+  not be found, so check the [WNPP bug list](https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=wnpp;dist=unstable) also)
+- [https://tracker.debian.org/](https://tracker.debian.org/)
+- [https://codesearch.debian.net/](https://codesearch.debian.net/)
+
 Setting up a local testing instance
 -----------------------------------