codecart

Case studies

Algorithms we took all the way into products

A selection of projects for semiconductor, mobile, automotive and imaging customers. Each one follows the same path: a demanding algorithm, a constrained platform, and code that shipped. Customer names are withheld where our contracts require it.

Audio · mobile
  • MATLAB → fixed-point C
  • multi-year · ongoing

Speaker boost & protection for mobile phones

A customer's proprietary algorithm lets small loudspeakers play louder while protecting them from damage. We turned it into production code that now ships in phones and other devices.

01Challenge

The algorithm existed as a MATLAB reference model. It had to become an efficient, portable software component that device makers could integrate on several processors.

02Constraints

Runs continuously in the audio path: real-time, low latency, tight cycle and memory budgets, several target processors — and the fixed-point code must preserve the behavior of the original model.

03What CODECart did

  • Converted the MATLAB reference algorithm to fixed-point
  • Wrote the fixed-point C reference implementation
  • Designed the module structure and the API
  • Built a set of elementary functions used as building blocks throughout the code
  • Designed the tests and validated each implementation
  • Ported and optimized the code for specific processors
  • Supported the customer and device makers during integration

04Platform & technology

MATLAB model → fixed-point C → processor-specific optimized code; audio DSPs and embedded processors.

05 Result

In production in multiple commercial mobile phones and other products. The collaboration has run for several years and is still ongoing.

Video · low-power vision processor
  • Movidius Myriad 2
  • H.264 / AVC

Multi-core H.264 encoder for a low-power vision processor

A high-performance H.264 encoder and decoder for a multi-core vision processor, flexible enough to serve very different products — from multi-camera surveillance to instant high-quality capture.

01Challenge

The processor vendor needed a video codec for its low-power multi-core chip that its customers could adapt to their own products without sacrificing performance.

02Constraints

Low-power, parallel architecture; real-time encoding of several inputs; and for burst mode, instant high quality instead of the strict bitrate constraints typical for video encoders.

03What CODECart did

  • Implemented a high-performance H.264 encoder optimized for an N-processor platform
  • Designed proprietary rate-control, motion-estimation and mode-decision schemes for parallel encoding
  • Built a highly optimized H.264 decoder for large video sizes
  • Delivered a very light API for real-time encoding of processed data from multiple sources
  • Delivered a lower-level custom API for burst encoding of short sequences in perfect quality — e.g. extracting still images from a video stream

04Platform & technology

Movidius Myriad 2 multi-core vision processor; H.264/AVC encoder and decoder; parallel encoding algorithms.

05 Result

Part of the software development kit shipped with the processor, and in commercial surveillance, security and imaging products — including those of a high-profile company.

DSP optimization · audio
  • NXP CoolFlux DSP / BSP
  • FPGA

Codecs & applications for an ultra-low-power audio DSP

For NXP's CoolFlux ultra-low-power DSP we built a portfolio of codecs, applications, libraries and benchmarks — and we still help the chip's customers build on it.

01Challenge

An ultra-low-power DSP is only as attractive as the software around it. Customers needed ready, efficient codecs and applications, credible benchmarks, and expert support.

02Constraints

An ultra-low-power DSP in battery-powered products: every cycle and every byte of memory counts, and all processing must meet real-time deadlines.

03What CODECart did

  • Implemented and optimized audio codecs for the CoolFlux DSP and BSP
  • Implemented and optimized the BDTI benchmarks for the CoolFlux DSP
  • Built embedded applications: an 8-channel 24-bit digital sound controller, AM/FM/RDS radio, an asynchronous sample-rate converter, G.722
  • Built an FM/DAB radio demonstrator on an FPGA board with three CoolFlux cores
  • Developed speech-enhancement and baseband-communication libraries
  • Ported and optimized customer algorithms; embedded compression
  • Provide ongoing support to the chip's customers

04Platform & technology

NXP CoolFlux DSP and BSP; multi-core FPGA prototype; audio codecs, radio, speech and baseband processing.

05 Result

Highly optimized components included in the platform SDK and used in many customers' products. We remain part of the support team, answering customers' questions and offering solutions.

Video · mobile platform
  • NXP / ST-Ericsson
  • 5 years

Imaging subsystem for a cellular platform

We designed and maintained the video part of the NXP / ST-Ericsson “Nexperia Cellular System Solution” — the complete path from camera to display — for a platform used by a major mobile phone company.

01Challenge

A mobile platform needed a complete, reliable imaging and video subsystem, maintained and extended across product generations.

02Constraints

A mobile SoC mixing hardware and software codecs; strict integration, quality and support requirements from a platform shipped to a handset maker.

03What CODECart did

  • Service layer for acquisition, encoding, decoding, rendering and display
  • Codec layer: video and image encoders, decoders and rendering
  • Driver layer controlling the camera, hardware encoder/decoder, hardware rendering and display
  • Specified the APIs and interactions between drivers, codecs and services
  • New features, code optimization, testing and validation
  • Level-3 customer support

04Platform & technology

Mobile SoC with hardware video IP; camera and display drivers; HW/SW codec partitioning.

05 Result

Five years of development and maintenance on a platform used in the handsets of a major mobile phone company.

Edge AI · vision
  • parallel vision processor
  • CNN

CNN optimization for a parallel vision processor

CNN-based applications, optimized for a parallel vision processor and packaged so that application developers can use them directly.

01Challenge

A platform vendor needed convolutional neural networks to run efficiently on its parallel processor, as part of the tools it offers developers.

02Constraints

Parallel architecture; the latency and power limits of embedded vision; clean interfaces for application developers.

03What CODECart did

  • Developed CNN-based applications
  • Optimized the network for the target parallel platform
  • Implemented the interfacing with the CNN
  • Integrated the CNN components into the vendor's software

04Platform & technology

Parallel vision processor; convolutional neural networks; embedded vision pipeline.

05 Result

The optimized CNN was integrated into an SDK and a toolkit available on the market.

More projects

Other work, briefly

Embedded

Wireless charging software

Customized embedded software solutions for NXP Smart Power, applied in wireless charging.

Signal

Haptics

Porting and optimization of applications for haptic technologies.

Vision

Stereo 3D meshing

An application for 3D meshing with a ZED stereo camera on an NVIDIA Jetson TX1 platform.

System

Interactive museum exhibit

A complete engineering solution based on an array of Raspberry Pi computers for an interactive museum exhibition.

Middleware

Imaging middleware

Development, maintenance and training around imaging middleware, with Ausy-Pentalog.

Research

ARTEMOS (EU-funded)

Participation in a European research project on agile RF transceivers for future radio communication.

Technology heritage

Two decades of video coding

The team formed in 2001 around video compression and joined Philips Semiconductors in 2002, as H.264/AVC was being standardized. That work — at the scale of a global semiconductor company — shaped the engineering method we use today.

H.264 encoder / decoder stack

  • All profiles: Baseline, Main, Extended, High; all levels
  • Multiple motion estimators, mode-decision and rate-control schemes
  • Multi-processor and HW/SW-partitioning ready

H.265 / HEVC stack

  • Encoder and decoder, all levels
  • Multiple motion estimators, mode-decision and rate-control schemes
  • Designed for scalability and customer adaptation
  • Philips · NXP · STEDesigned and implemented the H.264 code stack used by Philips, NXP and ST-Ericsson in products and research across several domains.
  • Mobile & set-top boxPorted and optimized the H.264 decoder on ARM for mobile phones, and integrated it into the SDK of TriMedia development boards for set-top boxes.
  • Video telephonyPorted and optimized the H.264 encoder for TriMedia, ARM, PC and a proprietary video-telephony system for a major consumer-electronics company.
  • HW/SW partitioningH.264 and VC-1 integration using hardware/software partitioning on NXP and ST-Ericsson IP.
  • Instruction setResearched and proposed new TriMedia instructions dedicated to H.264, included in the PNX1700.
  • IndustryH.264 technology preview and AVC promotion CD at IBC 2002 and 2004; patent submissions and papers.

Have an algorithm that must run in real time on constrained silicon?

Tell us about the target, the budget and the deadline. You will be talking to a senior engineer from the first call.

Discuss an engineering challenge or write to office@codec-art.com