且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

使用 apache nifi 使用预定义文件将列添加到 csv

更新时间:2023-12-01 11:40:04

使用

生成流文件:

更新记录:

配置 CSVReader 以将第一行视为标题.保持其他属性不变.配置 CSVRecordSetWrite 以将第一行视为标题,从架构文本属性派生架构并将架构文本设置为:

{"类型":"记录","name":"foobar","namespace":"my.example",领域":[{"姓名":"姓名",类型":字符串"},{姓名年龄",类型":整数"},{"name":"id",类型":字符串"},{"name":"尼克",类型":字符串"}]}

请注意,它包含新列.ReplaceTextWithMapping:

映射文件内容:

1 1S2 3S3 4S

值由制表符分隔.正则表达式必须匹配每行中没有后跟逗号的最后一个值:

[0-9](?!,)

I get a raw csv file which looks like this

id,name,star
1,sachith,2
2,nalaka,1
3,abc,3

I want to map star column with another file where it has

1  1S
2  3S
3  5S

and finally csv should look like

id,name,star,level
1,sachith,2,3S
2,nalaka,1,1S
3,abc,3,5S

I have used ReplaceTextWithMapping, but it replaces all the 1,2,3 values including in id column.

Here it defines replacing a value, but I want to map and add a new column to the record.

Edit:

After @Upvote's answer. My ReplaceTextWithMapping conf

Use ReplaceTextWithMapping. Overall flow:

GenerateFlowFile:

UpdateRecord:

Configure CSVReader to treat first line as header. Leave other properties untouched. Configure CSVRecordSetWrite to treat first line as header, schema to be derived from schema text property and set schema text to:

{
   "type":"record",
   "name":"foobar",
   "namespace":"my.example",
   "fields":[
      {
         "name":"name",
         "type":"string"
      },
      {
         "name":"age",
         "type":"int"
      },
      {
         "name":"id",
         "type":"string"
      },
      {
         "name":"nick",
         "type":"string"
      }
   ]
}

Notice that it includes the new column. ReplaceTextWithMapping:

Mapping file content:

1   1S
2   3S
3   4S

Values are separated by tab. Regex must match the last value not followed by a comma in each line:

[0-9](?!,)

Debuggex Demo

Result: