LENS documentation

Welcome to the LENS (Locational Encoding with Neuromorphic Systems) documentation. LENS is an event-based, neuromorphic visual place recognition system to perform accurate, fast, and energy-efficient robotic localization.

LENS was developed for the SynSense SPECK™, a combined neuromorphic processor and integrated dynamic vision sensor for ultra energy-efficient and compact place recognition. However, LENS is fully compatible with conventional compute hardware for benchmark and custom event-based localization datasets.

To get started, follow the steps below in Getting Started to install and run the demo for LENS.

Getting Started:

• Installation
  • Packages and dependencies
  • Repository cloning
• Quick start
  • Demo
  • Evaluation network
  • Training models
  • Direct Python execution
• Attributions
  • License
  • Citation
  • Acknowledgements

Model Training:

• Overview
  • Background
  • Connections
  • Weight updates
• Training Parameters
• Training New Models
  • Preparing your data
  • Ground truth file
  • Running the training
  • Network setup
  • Hyperparameters
  • Saving trained models

Model Evaluation:

• Overview
  • Time-based frame conversion
  • Constant leak integrate and fire neuron
  • Sequence matching
  • Evaluation metrics
• Evaluation Parameters
• Evaluating Models
  • Running the evaluation network
  • Ground truth tolerances
  • Sequence matching
  • Results output & saving
  • Additional evaluations

Optimization:

• Overview
• Using the optimizer
  • Running the optimizer script
  • Hyperparameters
  • Good Practices
  • Setting up Hyperparameter Sweeps
  • Wandb Sweep Setup
  • Sweep Method Configuration
  • Model Evaluation
  • Custom Metrics (e.g., Recall@1)

SPECK deployment:

• Overview
• Speck Deployment Parameters
• Create a SPECK dataset
  • Collect events for images
  • Modify collection window
  • Set the experimental details
  • Dataset considerations
• Deploying to SPECK
• On-device localization
  • Online sequence matching
  • Power measurements
  • Saving input event streams

Modules Reference:

• LENS package
  • Subpackages
  • Submodules
  • LENS.collect_data module
  • LENS.run_model module
  • LENS.run_speck module
  • LENS.train_model module
  • Module contents
• LENS.src package
  • Submodules
  • LENS.src.blitnet module
  • LENS.src.dataset module
  • LENS.src.demo module
  • LENS.src.loggers module
  • LENS.src.match module
  • LENS.src.metrics module
  • LENS.src.sad module
  • LENS.src.speck2f module
  • Module contents
• LENS.tools package
  • Submodules
  • LENS.tools.checker module
  • LENS.tools.create_GTtol module
  • LENS.tools.create_data_csv module
  • LENS.tools.dvstools module
  • LENS.tools.manual_eventframe_generator module
  • LENS.tools.matching_plots module
  • LENS.tools.plot_results module
  • LENS.tools.process_dvs module
  • LENS.tools.read_gps module
  • LENS.tools.rename_images module
  • Module contents