(1)一段 建表语句:
[ROW FORMAT row_format] [STORED AS file_format] | STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)] -- (Note: Available in Hive 0.6.0 and later) ] row_format : :DELIMITED [FIELDS TERMINATED BY char [ESCAPED BY char]] [COLLECTION ITEMS TERMINATED BY char] [MAP KEYS TERMINATED BY char] [LINES TERMINATED BY char] [NULL DEFINED AS char] -- (Note: Available in Hive 0.13 and later) | SERDE serde_name [WITH SERDEPROPERTIES (property_name=property_value, property_name=property_value, ...)] file_format: : SEQUENCEFILE | TEXTFILE -- (Default, depending on hive.default.fileformat configuration) | RCFILE -- (Note: Available in Hive 0.6.0 and later) | ORC -- (Note: Available in Hive 0.11.0 and later) | PARQUET -- (Note: Available in Hive 0.13.0 and later) | AVRO -- (Note: Available in Hive 0.14.0 and later) | INPUTFORMAT input_format_classname OUTPUTFORMAT output_format_classname
(2)根据建表语句逐层展开:hive的存储形式,引申思考下行式存储、列式存储的 在读取时的优劣
hive表数据在存储在文件系统上的,因此需要有文件存储格式来规范化数据的存储,一边hive写数据或者读数据。hive有一些已构建好的存储格式,也支持用户自定义文件存储格式。主要由两部分内容构成file_format和row_format,两者息息相关。
(3)文件格式:TEXTFILE,SEQUENCEFILE,RCFILE 及 自定义输入格式DuallnputFormat
(4)记录格式:SerDe
(5)CSV和TSV SerDe
SerDe is a short name for "Serializer and Deserializer."Hive uses SerDe (and FileFormat) to read and write table rows.HDFS files --> InputFileFormat -->--> Deserializer --> Row objectRow object --> Serializer --> --> OutputFileFormat --> HDFS files
load数据时 根据表的文件格式及分割方式 直接写入存储,读时校验数据
hive的读时模式 与 传统关系型数据库写入模式的差异