Roadside incident reporting is the process of recording information of a reported incident and creating a summary report that accurately represents the situation for effective resolution and case review.
- Develop a web-based prototype application for the purpose of streamlining and/or automating aspects of the roadside incident report handling process for operational agents.
- Take live audio input and transcribe it to highlight key information with a summary.
- Use mapping tools to allow live location information delivery related to incoming calls.
Speech Processing & Text Analysis
The speech module was created to process audio input for automatic transcription. Using Microsoft Azure Cognitive Speech services an audio stream is sent for processing in the cloud and continuously returns a transcription that is then used with Azures Text Analytic services to recognize, categorize, and extract keywords for the creation of a summary report.
Mapping & Location Technologies
The final prototype combines the two individual modules mentioned previously to allow for autonomous map updates during live caller transcription and report generation, adding a marker for the greatest match and providing suggested relevant locations for the operator to filter at their discretion. In turn allowing for the operator to better focus on incident management instead of information processing.
Below is a screenshot of the final prototype and the demonstration video.
In order to assert the applications proper functionality both black and white box testing were undertaken at multiple stages of development. On top of this, performance testing was undertaken to compare the effectiveness of the base text analytic services offered by Azure and the addition of a keyword phrase list used to potentially increase recognition of technical terms.
The figures below show that whilst the inclusion of a phrase list can increase recognition of fixed technical terms it should be replaced with the training of a custom speech model for the best performance due to increased flexibility in recognition of proximally similar key phrases, as well as the benefit of tailoring for regional dialects and accents.
Issues & Challenges
The following video highlights the issues faced and the key decisions made during the project.
The following video highlights the future possibilities for the project given further time and access to existing systems.
BSc(Hons) Computer Science
I am a final year student at Teesside University aiming to graduate in the summer of 2021 with a predicted first grade degree in BSc (Hons) Computer Science. I undertook a 16 month placement at the Wilton Business Center in Redcar with Sabisu/Aspentech in which I was seconded to SABIC, one of the worlds largest petrochemical manufacturers, during which I co-managed a queue of support tickets and project work ranging from critical system updates to the creation of new reporting tools. In my free time I enjoy thinking about game development and playing games both social and solitary.
Visual Studio, Net Beans, Azure Cloud Services, Git, Visio, Android Studio, Microsoft Office
Driving License, Completed Fire Team Project with Cleveland Fire Brigade
Employment, Work Experience & Volunteering
Relevant Work Experience
ASSOCIATE CONSULTANT ENGINEER • SABISU/ASPENTECH (STUDENT PLACEMENT) • 2019 (JUN) – 2020 (SEP)
PASS Leader in Java at Teesside University during 2nd year, with an open mind in further pursuing teaching or tutoring.
Vice-chair of Finance and Co-Founder of the Teesside University Magic the Gathering society.
Actively interacted with students and staff, providing troubleshooting and feedback in key areas, as part of my responsibilities as a 2nd year Computer Science course representative.
Volunteered for Scope charity in Redcar (2015)
Board games, PC games, card games (Magic the Gathering), table top role playing games (Dungeons & Dragons), table top war games (Warhammer 40k), fiction (Sci-fi, Fantasy), cycling, game development, mental health & self improvement.