Convert XLSX to Avro

XLSX files are Excel files that use the Office Open XML format, which Microsoft introduced with Excel 2007.

Like XLS files, XLSX files can have multiple sheets, and each sheet can contain multiple rows and columns of data. They offer several advantages over the older XLS format.

Key features of XLSX files include:

• Better compression, resulting in smaller file sizes
• Enhanced security features, including password protection and digital signatures
• Better recovery from file corruption
• Support for more rows (1,048,576) and columns (16,384) per sheet
• Works with a wide range of applications and programming languages

XLSX files are based on XML and ZIP technologies, making them easier for third-party software to read and write.

Avro is a row-based data serialization system developed within Apache's Hadoop project. It provides rich data structures and a compact, fast binary data format.

Technical Details

Avro uses JSON for defining data schemas, which are stored with the data. This enables schema evolution while maintaining compatibility. The data itself is stored in a compact binary format.

Advantages

• Compact binary serialization
• Schema definition included with the data
• Support for schema evolution
• Dynamic typing and code generation

Limitations

• Not human-readable without special tools
• Less widely supported than formats like JSON or CSV
• More complex to implement than simpler formats
• Less efficient for columnar queries than Parquet