Coding Period Week 2

Week 2 Progress (June 3 - June 9)

This week was completely focused on integrating the post-scan plugin --package-summary for the computation of package-level summaries. These plugins primarily handle data processing, summarization, and reporting, relying on results from all scan plugins. They introduce new attributes at the codebase or file level, and may also remove or modify data as needed for consolidation or summarization. The base plugin class to be extended is PostScanPlugin, located at /src/plugincode/post_scan.py.

Previously, the scancode-toolkit had --summary as the post-scan plugin for summarizing the entire codebase. The available post-scan options in scancode-toolkit include:

• --license-clarity-score: Computes a summary license clarity score at the codebase level. This is a sub-option of --classify.
• --summary: Summarizes scans by providing declared origin information and other detected information at the codebase attribute level. When --summary plugin is applied, the JSON file is structured as below-

  {
  "headers": [...],
  "summary": {
    "declared_license_expression": null,
    "license_clarity_score": {...},
    "declared_holder": "",
    "primary_language": "C",
    "other_license_expressions": [...],
    "other_holders": [...]
    "other_languages": [...]
  },
  "files": [...]
  }
  

The ScanCode toolkit utilizes the plugincode for its pluggable functionality, including Click plugins. This plugincode is built upon the pluggy package. To incorporate a post-scan plugin:

1. Begin by creating a new file named package_summary.py in the src/summarycode directory, which will contain all necessary components.

2. Import required modules:

    from plugincode.post_scan import PostScanPlugin
    from plugincode.post_scan import post_scan_impl
    from scancode import CommandLineOption
    from scancode import POST_SCAN_GROUP
   

3. Construct a PostScanPlugin class:

   • The PostScanPlugin class PostScanPlugin inherits from the CodebasePlugin class, which itself inherits from BasePlugin.

    @post_scan_impl
    class PackageSummary(PostScanPlugin):
        """
        Summary at the Package Level.
        """
   
        options = [
            PluggableCommandLineOption(('--package-summary',),
            is_flag=True, default=False,
            help='Generate Package Level summary',
            help_group=POST_SCAN_GROUP)
        ]
   
        def is_enabled(self, package_summary, **kwargs):
            return package_summary
   
        def process_codebase(self, codebase, package_summary, **kwargs):
            """
            Process the codebase.
            """
            if not self.is_enabled(package_summary):
                return
   

4. Add the entry point package_summary in the setup.cfg file:

    scancode_post_scan =
        package_summary = summarycode.package_summary:PackageSummary
        summary = summarycode.summarizer:ScanSummary
        tallies = summarycode.tallies:Tallies
        tallies-with-details = summarycode.tallies:TalliesWithDetails
   

   • here, package_summary refers to the name of the command flag.
   • summarycode denotes the directory containing the plugin file.
   • the package_summary is the name of the Python file housing the plugin.
   • PackageSummary is the name of the PostScanPlugin created.

5. Loaded the Plugin

   • After running the command

     pip install .
     

   • This ensures that the plugin is successfully installed.

Discussions

During the Weekly Status Call, we discussed with the mentors about adding small unit tests for the Post Scan Plugin. Additionally, there was a discussion about the Pull Request that needs to be updated with the failing tests regenerated to ensure all tests pass successfully.

Philippe also provided insights into various examples of monorepos and scenarios where a package contains more than one package within it. He emphasized the importance of developing appropriate populating methods to handle such cases effectively.

Conclusion and Further Plans

Now that we have the plugin, we discussed further plans to develop methods for populating the License Clarity Field and other fields for Package Level scanning.

Attendees

• Philippe Ombredanne
• Ayan Sinha Mahapatra
• Jono Yang
• Omkar Phansopkar
• Ziad Hany
• Keshav Priyadarshi
• Michael Ehab
• Pranay Das
• Ambuj Kulshreshtha
• Swastik Sharma

Visitor Count