且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

使用Node将Stream上传的文件流到Azure blob存储

更新时间:2023-02-09 08:27:50

解决方案(基于与@danielepolencic的讨论)



如果我们禁用了Express的bodyparser()中间件,请使用Multiparty(npm install multiparty)(一个Formidable的分支),可以访问multipart数据(有关更多信息,请参阅他们关于此操作的注释)。不同于Formidable,多方不会将文件流传输到磁盘,除非你告诉它。

  app.post('/ upload' function(req,res){
var blobService = azure.createBlobService();
var form = new multiparty.Form();
form.on('part',function(part) {
if(part.filename){

var size = part.byteCount - part.byteOffset;
var name = part.filename;

blobService.createBlockBlobFromStream('c',name,part,size,function(error){
if(error){
res.send({Grrr:error});
}
$);
} else {
form.handlePart(part);
}
});
form.parse(req);
res.send('OK');
});

支持@danielepolencic帮助找到解决方案。


Using Express with Node, I can upload a file successfully and pass it to Azure storage in the following block of code.

app.get('/upload', function (req, res) {
    res.send(
    '<form action="/upload" method="post" enctype="multipart/form-data">' +
    '<input type="file" name="snapshot" />' +
    '<input type="submit" value="Upload" />' +
    '</form>'
    );
});

app.post('/upload', function (req, res) {
    var path = req.files.snapshot.path;
    var bs= azure.createBlobService();
    bs.createBlockBlobFromFile('c', 'test.png', path, function (error) { });
    res.send("OK");
});

This works just fine, but Express creates a temporary file and stores the image first, then I upload it to Azure from the file. This seems like an inefficient and unnecessary step in the process and I end up having to manage cleanup of the temp file directory.

I should be able to stream the file directly to Azure storage using the blobService.createBlockBlobFromStream method in the Azure SDK, but I am not familiar enough with Node or Express to understand how to access the stream data.

app.post('/upload', function (req, res) {

    var stream = /// WHAT GOES HERE ?? ///

    var bs= azure.createBlobService();
    bs.createBlockBlobFromStream('c', 'test.png', stream, function (error) { });
    res.send("OK");
});

I have found the following blog which indicates that there may be a way to do so, and certainly Express is grabbing the stream data and parsing and saving it to the file system as well. http://blog.valeryjacobs.com/index.php/streaming-media-from-url-to-blob-storage/

vjacobs code is actually downloading a file from another site and passing that stream to Azure, so I'm not sure if it can be adapted to work in my situation.

How can I access and pass the uploaded files stream directly to Azure using Node?

SOLUTION (based on discussion with @danielepolencic)

Using Multiparty(npm install multiparty), a fork of Formidable, we can access the multipart data if we disable the bodyparser() middleware from Express (see their notes on doing this for more information). Unlike Formidable, Multiparty will not stream the file to disk unless you tell it to.

app.post('/upload', function (req, res) {
    var blobService = azure.createBlobService();
    var form = new multiparty.Form();
    form.on('part', function(part) {
        if (part.filename) {

            var size = part.byteCount - part.byteOffset;
            var name = part.filename;

            blobService.createBlockBlobFromStream('c', name, part, size, function(error) {
                if (error) {
                    res.send({ Grrr: error });
                }
            });
        } else {
            form.handlePart(part);
        }
    });
    form.parse(req);
    res.send('OK');
});

Props to @danielepolencic for helping to find the solution to this.