本次,我们来实现一个单个大文件上传,并且把后台对上传文件的处理进度通过ASP.NET CORE SignalR反馈给前端展示,比如上传一个大的zip压缩包文件,后台进行解压缩,并且对压缩包中的文件进行md5校验,同时要求前台可以实时(实际情况看网络情况)展示后台对压缩包的处理进度(解压、校验文件)。
在前端上传文件的组件选择上,采用了WebUploader(http://fex.baidu.com/webuploader/)这个优秀的前端组件,下面是来自它的官网介绍:
WebUploader是由Baidu WebFE(FEX)团队开发的一个简单的以HTML5为主,FLASH为辅的现代文件上传组件。在现代的浏览器里面能充分发挥HTML5的优势,同时又不摒弃主流IE浏览器,沿用原来的FLASH运行时,兼容IE6+,iOS 6+, android 4+。两套运行时,同样的调用方式,可供用户任意选用。
采用大文件分片并发上传,极大的提高了文件上传效率。
WebUploader的功能很多,本次只使用它的上传前文件MD5校验、并发分片上传、分片MD5校验三个主要功能,分别来实现类似网盘中的文件【秒传】,浏览器多线程上传文件和文件的断点续传。
阅读参考此文章前,请先看一下https://www.cnblogs.com/wdw984/p/14645614.html
此文章是上一篇的功能扩展,一些基本的程序模块逻辑都已经在上一篇文章中做了介绍,这里就不再重复。
在正式使用WebUploader进行上传文件之前,先对它的执行流程和触发的事件做个大致的介绍(如有不对的地方请指正),我们可以通过它触发的事件来做相应的流程或业务上的预处理,比如文件秒传,重复文件检测等。
当WebUploader正确加载完成后,会触发它的ready事件;
当点击文件选择框的时候(其它方式传入文件所触发的事件请参考官方文档),会触发它的dialogOpen事件;
当选择文件完成后,触发事件的流程为:beforeFileQueued ==> fileQueued ==> filesQueued;
当点击(开始)上传的时候,触发事件的流程为:
1、正常文件上传流程
startUpload(如秒传(后台通过文件的md5判断返回)秒传则触发UploadSkip) ==> uploadStart ==> uploadBeforeSend ==> uploadProgress ==> uploadAccept(接收服务器处理分块传输后的返回信息) ==> uploadSuccess ==> uploadComplete ==> uploadFinished
2、文件秒传或续传流程
startUpload ==> uploadStart(触发秒传或文件续传) ==> uploadSkip ==> uploadSuccess ==> uploadComplete ==> uploadFinished
现在,我们在上一次项目的基础上做一些改造升级,最终实现我们本次的功能。
先看效果(GIF录制时间略长,请耐心等待一下)
首先,我们引用大名鼎鼎的WebUploader组件库。在项目上右键==>添加==>客户端库 的界面中选择unpkg然后输入webuploader
为了实现压缩文件的解压缩操作,我们在Nuget中引用SharpZipLib组件
然后我们在appsettings.json中增加一个配置用来保存上传文件。
1 {
2 "Logging": {
3 "LogLevel": {
4 "Default": "Information",
5 "Microsoft": "Warning",
6 "Microsoft.Hosting.Lifetime": "Information"
7 }
8 },
9 "FileUpload": {
10 "TempPath": "temp",//临时文件保存目录
11 "FileDir": "upload",//上传完成后的保存目录
12 "FileExt": "zip,rar"//允许上传的文件类型
13 },
14 "AllowedHosts": "*"
15 }
在项目中新建一个Model目录,用来实现上传文件的相关配置,建立相应的多个类文件
FileUploadConfig.cs 服务器用来接受和保存文件的配置
1 using System;
2
3 namespace signalr.Model
4 {
5 ///
8 [Serializable]
9 public class FileUploadConfig
10 {
11 ///
14 public string TempPath { get; set; }
15 ///
18 public string FileDir { get; set; }
19 ///
22 public string FileExt { get; set; }
23 }
24 }
UploadFileWholeModel.cs 前台开始传输前会对文件进行一次MD5算法,这里可以通过文件MD5值传递给后台来通过比对已上传的文件MD5值列表来实现秒传功能
1 namespace signalr.Model
2 {
3 ///
6 public class UploadFileWholeModel
7 {
8 ///
11 public string CheckType { get; set; }
12 ///
15 public string FileMd5 { get; set; }
16 ///
19 public string FileGuid { get; set; }
20 ///
23 public string FileName { get; set; }
24 ///
27 public int? FileSize { get; set; }
28 }
29 }
UploadFileChunkModel.cs 前台文件分块传输的时候会对分块传输内容进行MD5计算,并且分块传输的时候会传递当前分块的一些信息,这里对应的后台接收实体类。
我们可以通过分块传输的MD5值来实现文件续传功能(如文件的某块MD5已存在则返回给前台跳过当前块)
1 namespace signalr.Model
2 {
3 ///
6 public class UploadFileChunkModel
7 {
8 ///
11 public string CheckType { get; set; }
12 ///
15 public long? FileSize { get; set; }
16 ///
19 public string FileId { get; set; }
20 ///
23 public long? ChunkStart { get; set; }
24 ///
27 public long? ChunkEnd { get; set; }
28 ///
31 public long? ChunkSize { get; set; }
32 ///
35 public string ChunkIndex { get; set; }
36 ///
39 public string ChunkCount { get; set; }
40 ///
43 public string ChunkId { get; set; }
44 ///
47 public string Md5 { get; set; }
48 }
49 }
FormData.cs 这是分块传输时传递的当前块的信息配置
1 using System;
2
3 namespace signalr.Model
4 {
5 ///
8 [Serializable]
9 public class FormData
10 {
11 ///
14 public string Checktype { get; set; }
15 ///
18 public int? Filesize { get; set; }
19 ///
22 public string Fileid { get; set; }
23 ///
26 public int? Chunksize { get; set; }
27 ///
30 public int? Chunkindex { get; set; }
31 ///
34 public int? Chunkstart { get; set; }
35 ///
38 public int? Chunkend { get; set; }
39 ///
42 public int? Chunkcount { get; set; }
43 ///
46 public string Chunkid { get; set; }
47 ///
50 public string Md5 { get; set; }
51 }
52 }
UploadFileModel.cs 每次上传文件的时候,前台都会传递这些参数给服务器,服务器可以根据参数做相应的处理
1 using System;
2 using Microsoft.AspNetCore.Mvc;
3
4 namespace signalr.Model
5 {
6 ///
9 [Serializable]
10 public class UploadFileModel
11 {
12 ///
15 public string Id { get; set; }
16 ///
19 public string FileMd5 { get; set; }
20 ///
23 public string Chunk { get; set; }
24 ///
27 public string Name { get; set; }
28 ///
31 [FromForm(Name = "type")]
32 public string FileType { get; set; }
33 ///
36 public long? Size { get; set; }
37 ///
40 public string Guid { get; set; }
41 ///
44 public FormData FromData { get; set; }
45 ///
48 public byte[] FileData { get; set; }
49 }
50 }
UploadFileMergeModel.cs 当所有块传输完成后,传递给后台一个合并文件的请求,后台通过参数中的信息把分块保存的文件合并成一个完整的文件
1 namespace signalr.Model
2 {
3 ///
6 public class UploadFileMergeModel
7 {
8 ///
11 public string CheckType { get; set; }
12 ///
15 public long? FileSize { get; set; }
16 ///
19 public int? ChunkNumber { get; set; }
20 ///
23 public string FileMd5 { get; set; }
24 ///
27 public string FileName { get; set; }
28 ///
31 public string FileExt { get; set; }
32 }
33 }
为了实现【秒传】和分块传输时的【断点续传】功能,我们在Class目录中定义一个UploadFileList.cs类,用来模拟持久化保存服务器所接收到的文件MD5校验列表和已接收的分块MD5值信息,这里我们使用了并发线程安全的ConcurrentDictionary和ConcurrentBag
1 using System;
2 using System.Collections.Concurrent;
3
4 namespace signalr.Class
5 {
6 public class UploadFileList
7 {
8 private static readonly Lazy
9 private static readonly Lazy
10 new Lazy
11 public UploadFileList()
12 {
13 ServerUploadFileList = _serverUploadFileList;
14 UploadChunkFileList = _uploadChunkFileList;
15 }
16
17 ///
20 public readonly Lazy
21 ///
24 public readonly Lazy
25 }
26 }
扩展一下HubInterface/IChatClient.cs 用来推送给前台展示后台处理的信息
public interface IChatClient
{
///
/// 消息实体类
///
Task ReceiveMessage(ClientMessageModel clientMessageModel);
///
/// JSON格式的可以被Echarts识别的data数据
///
Task EchartsMessage(Array data);
///
/// 消息实体类
///
Task GetMyId(ClientMessageModel clientMessageModel);
///
/// 消息实体类
///
Task UploadInfoMessage(ClientMessageModel clientMessageModel);
}
扩展一下Class/ClientMessageModel.cs
/// <summary>
/// 服务端发送给客户端的信息
/// </summary>
\[Serializable\]
public class ClientMessageModel
{
/// <summary>
/// 接收用户编号
/// </summary>
public string UserId { get; set; }
/// <summary>
/// 组编号
/// </summary>
public string GroupName { get; set; }
/// <summary>
/// 发送的内容
/// </summary>
public string Context { get; set; }
/// <summary>
/// 自定义的响应编码
/// </summary>
public string Code { get; set; }
}
我们在Startup.cs中注入上传文件的配置,同时把前文的XSRF防护去掉,我们在前台请求的时候带上防护认证信息。
public void ConfigureServices(IServiceCollection services)
{
services.AddSignalR();
services.AddRazorPages()
services.AddSingleton
services.AddOptions()
.Configure
}
在项目的wwwroot/js下新建一个uploader.js
"use strict";
var connection = new signalR.HubConnectionBuilder()
.withUrl("/chatHub")
.withAutomaticReconnect()
.configureLogging(signalR.LogLevel.Debug)
.build();
var user = "";
connection.on("GetMyId", function (data) {
user = data.userId;
});
connection.on("ReceiveMessage", function (data) {
console.log(data.userId + data.context);
});
connection.on("UploadInfoMessage", function (data) {
switch (data.code) {
case "200":
$('.modal-body').append($("
" + data.context + "
"));//当后台返回处理完成或出错时,前台显示内容,同时显示关闭按钮" + data.context + "
"));//展示后台返回信息" + data.context + "
"));connection.start().then(function () {
console.log("服务器已连接");
}).catch(function (err) {
return console.error(err.toString());
});
在项目的Pages/Shared中新建一个Razor布局页_LayoutUpload.cshtml
在Pages目录下新建一个upload目录,然后在它下面新建一个index.cshtml,这个文件中实现了Webuploader中我们所要使用的事件监测、文件上传功能。
1 @page "{handler?}"
2 @model MediatRStudy.Pages.upload.IndexModel
3 @{
4 ViewBag.Title = "WebUploader";
5 Layout = "_LayoutUpload";
6 }
7 @section Scripts
8 {
9
10
11
12
327 }
328
index.cshtml的代码文件如下
本示例只能解压缩zip文件,并且密码是123456,友情提示,不要用QQ浏览器调试,否则会遇到选择文件后DEBUG停止运行。
本示例只能解压缩zip文件,并且密码是123456,友情提示,不要用QQ浏览器调试,否则会遇到选择文件后DEBUG停止运行。****
本示例只能解压缩zip文件,并且密码是123456,友情提示,不要用QQ浏览器调试,否则会遇到选择文件后DEBUG停止运行。****
1 using ICSharpCode.SharpZipLib.Zip;
2 using Microsoft.AspNetCore.Http;
3 using Microsoft.AspNetCore.Mvc;
4 using Microsoft.AspNetCore.Mvc.RazorPages;
5 using Microsoft.AspNetCore.SignalR;
6 using Microsoft.Extensions.Options;
7 using signalr.Class;
8 using signalr.HubInterface;
9 using signalr.Hubs;
10 using signalr.Model;
11 using System;
12 using System.Collections.Concurrent;
13 using System.Diagnostics;
14 using System.IO;
15 using System.Linq;
16 using System.Text.Json;
17 using System.Threading.Tasks;
18
19 namespace signalr.Pages.upload
20 {
21 public class IndexModel : PageModel
22 {
23 private readonly IOptionsSnapshot
24 private readonly IOptionsSnapshot
25 private readonly string[] _fileExt;
26 private readonly IHubContext
27 public IndexModel(IOptionsSnapshot
28 {
29 _fileUploadConfig = fileUploadConfig;
30 _fileList = fileList;
31 _fileExt = _fileUploadConfig.Value.FileExt.Split(',').ToArray();
32 _hubContext = hubContext;
33 }
34 public IActionResult OnGet()
35 {
36 ViewData["Token"] = "666";
37 return Page();
38 }
39
40 #region 上传文件
41
42 ///
45 ///
46 public async Task
47 {
48 if (_fileUploadConfig.Value == null)
49 {
50 return new JsonResult(new { code = 400, msg = "服务器配置不正确" });
51 }
52
53 if (file == null || file.Length < 1)
54 {
55 return new JsonResult(new { code = 404, msg = "没有接收到要保存的文件" });
56 }
57 Request.EnableBuffering();
58 var formData = Request.Form["formData"];
59 if (model == null || string.IsNullOrWhiteSpace(formData))
60 {
61 return new JsonResult(new { code = 401, msg = "没有接收到必要的参数" });
62 }
63
64 var request = model;
65 long.TryParse(Request.Form["size"], out var fileSize);
66 request.Size = fileSize;
67 try
68 {
69 request.FromData = JsonSerializer.Deserialize
70 }
71 catch (Exception e)
72 {
73 Debug.WriteLine(e);
74 }
75
76 if (request.FromData == null)
77 {
78 return new JsonResult(new { code = 402, msg = "参数错误" });
79 }
80
81 #if DEBUG
82 Debug.WriteLine($"原文件名:{request.Name},文件编号:{request.Guid},文件块编号:{request.Chunk},文件Md5:{request.FileMd5},当前块UID:{request.FromData?.Chunkid},当前块MD5:{request.FromData?.Md5}");
83 #endif
84 var fileExt = request.Name.Substring(request.Name.LastIndexOf('.') + 1).ToLowerInvariant();
85 if (!_fileExt.Contains(fileExt))
86 {
87 return new JsonResult(new { code = 403, msg = "文件类型不在允许范围内" });
88 }
89 if (_fileList.Value.UploadChunkFileList.Value.ContainsKey(request.Guid))
90 {
91 if (!_fileList.Value.UploadChunkFileList.Value[request.Guid].Any(x => string.Equals(x, request.FromData.Md5, StringComparison.OrdinalIgnoreCase)))
92 {
93 _fileList.Value.UploadChunkFileList.Value[request.Guid].Add(request.FromData.Md5);
94 }
95 #if DEBUG
96 else
97 {
98 Debug.WriteLine($"ContainsKey{request.FromData.Chunkindex}存在校验值{request.FromData.Md5}");
99 return new JsonResult(new { code = 200, msg = "成功接收", miaochuan = true });
100 }
101 #endif
102 }
103 else
104 {
105 return new JsonResult(new { code = 405, msg = "接收失败,因为服务器没有找到此文件的容器,请重新上传" });
106 }
107
108 var dirPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, _fileUploadConfig.Value.TempPath, request.Guid);
109 if (!Directory.Exists(dirPath))
110 {
111 Directory.CreateDirectory(dirPath);
112 }
113
114 var tempFile = string.Concat(dirPath, "\\", request.FromData.Chunkindex.ToString().PadLeft(4, '0'), ".", fileExt);
115 try
116 {
117
118 await using var fs = System.IO.File.OpenWrite(tempFile);
119 request.FileData = new byte[Convert.ToInt32(request.FromData.Chunksize ?? 0)];
120
121 await using var memStream = new MemoryStream();
122 await file.CopyToAsync(memStream);
123
124 request.FileData = memStream.ToArray();
125
126 await fs.WriteAsync(request.FileData, 0, request.FileData.Length);
127 await fs.FlushAsync();
128 }
129 catch (Exception e)
130 {
131 #if DEBUG
132 Debug.WriteLine($"White Error:{e}");
133 #endif
134 _fileList.Value.UploadChunkFileList.Value.TryRemove(request.Guid, out _);
135 }
136 return new JsonResult(new { code = 200, msg = "成功接收", miaochuan = false });
137 }
138
139 #endregion
140
141 #region 合并上传文件
142
143 ///
146 /// 前台传递的请求合并的参数
147 ///
148 public async Task
149 {
150 return await Task.Run(async () =>
151 {
152 if (mergeModel == null || string.IsNullOrWhiteSpace(mergeModel.FileName) ||
153 string.IsNullOrWhiteSpace(mergeModel.FileMd5))
154 {
155 return new JsonResult(new { code = 300, success = false, count = 0, size = 0, msg = "合并失败,参数不正确。" });
156 }
157 if (!_fileExt.Contains(mergeModel.FileExt.ToLowerInvariant()))
158 {
159 return new JsonResult(new { code = 403, success = false, msg = "文件类型不在允许范围内" });
160 }
161
162 var fileSavePath = "";
163 if (!_fileList.Value.ServerUploadFileList.Value.ContainsKey(mergeModel.FileMd5))
164 {
165 //合并块文件、删除临时文件
166 var chunks = Directory.GetFiles(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, _fileUploadConfig.Value.TempPath, mergeModel.FileName), "*.*");
167 if (!chunks.Any())
168 {
169 return new JsonResult(new { code = 302, success = false, count = 0, size = 0, msg = "未找到文件块信息,请重试。" });
170 }
171 var dirPath = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, _fileUploadConfig.Value.FileDir);
172 if (!Directory.Exists(dirPath))
173 {
174 Directory.CreateDirectory(dirPath);
175 }
176 fileSavePath = Path.Combine(_fileUploadConfig.Value.FileDir,
177 string.Concat(mergeModel.FileName, ".", mergeModel.FileExt));
178 await using var fs =
179 new FileStream(Path.Combine(dirPath, string.Concat(mergeModel.FileName, ".", mergeModel.FileExt)), FileMode.Create);
180 foreach (var file in chunks.OrderBy(x => x))
181 {
182 //Debug.WriteLine($"File==>{file}");
183 var bytes = await System.IO.File.ReadAllBytesAsync(file);
184 await fs.WriteAsync(bytes.AsMemory(0, bytes.Length));
185 }
186 //Directory.Delete(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, _fileUploadConfig.Value.TempPath, mergeModel.FileName), true);
187
188
189 if (!_fileList.Value.ServerUploadFileList.Value.TryAdd(mergeModel.FileMd5, fileSavePath))
190 {
191 return new JsonResult(new { code = 301, success = false, count = 0, size = 0, msg = "服务器保存文件失败,请重试。" });
192 }
193 }
194 var user = Request.Headers["userid"];
195 //调用解压文件
196 if (string.Equals(mergeModel.FileExt.ToLowerInvariant(), "zip"))
197 {
198 DoUnZip(Path.Combine(AppDomain.CurrentDomain.BaseDirectory, fileSavePath), user.ToString());
199 }
200 else
201 {
202 await SentMessage(user.ToString(), "服务器只能解压缩zip格式文件。", "200");
203 }
204 return new JsonResult(new { code = 200, success = true, count = 0, size = 0, msg = "上传成功", url = fileSavePath });
205 });
206
207 }
208
209 #endregion
210
211 #region 文件秒传检测、文件类型允许范围检测
212 public JsonResult OnPostFileWholeAsync(UploadFileWholeModel model)
213 {
214 if (model == null || string.IsNullOrWhiteSpace(model.FileMd5))
215 {
216 return new JsonResult(new { Code = 300, IsExist = false, success = false, FileUrl = "", Msg = "参数不正确" });
217 }
218 var fileExt = model.FileName.Substring(model.FileName.LastIndexOf('.') + 1).ToLowerInvariant();
219 if (!_fileExt.Contains(fileExt))
220 {
221 return new JsonResult(new { code = 403, success = false, msg = "文件类型不在允许范围内" });
222 }
223 if (_fileList.Value.ServerUploadFileList.Value.ContainsKey(model.FileMd5))
224 {
225 return new JsonResult(new { Code = 200, IsExist = true, success = true, FileUrl = _fileList.Value.ServerUploadFileList.Value[model.FileMd5], miaochuan = true });
226 }
227 //检测的时候创建待上传文件的分块MD5容器
228 _fileList.Value.UploadChunkFileList.Value.TryAdd(model.FileGuid, new ConcurrentBag
229
230 return new JsonResult(new { Code = 200, IsExist = false, FileUrl = "" });
231 }
232 #endregion
233
234 #region 文件块秒传检测
235 public JsonResult OnPostFileChunkAsync(UploadFileChunkModel model)
236 {
237 if (model == null || string.IsNullOrWhiteSpace(model.Md5) || string.IsNullOrWhiteSpace(model.FileId))
238 {
239 return new JsonResult(new { Code = 300, IsExist = false, success = false, FileUrl = "", Msg = "参数不正确" });
240 }
241
242 if (!_fileList.Value.UploadChunkFileList.Value.ContainsKey(model.FileId))
243 {
244 return new JsonResult(new { Code = 200, IsExist = false, FileUrl = "" });
245 }
246
247 if (!_fileList.Value.UploadChunkFileList.Value[model.FileId].Contains(model.Md5))
248 {
249 return new JsonResult(new { Code = 200, IsExist = false, FileUrl = "" });
250 }
251 return new JsonResult(new { Code = 200, IsExist = true, success = true, miaochuan = true });
252 }
253 #endregion
254
255 #region 解压、校验文件
256
257 private void DoUnZip(string zipFile, string user)
258 {
259 Task.Factory.StartNew(async () =>
260 {
261 if (!System.IO.File.Exists(zipFile))
262 {
263 //发送一条文件不存在的消息
264 await SentMessage(user, "访问上传的压缩包失败");
265 return;
266 }
267 var fastZip = new FastZip
268 {
269 Password = "123456",
270 CreateEmptyDirectories = true
271 };
272 try
273 {
274 var zipExtDir = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "ZipEx", "601018");
275 //删除现有文件夹
276 if (Directory.Exists(zipExtDir))
277 Directory.Delete(zipExtDir, true);
278 //发送开始解压缩信息
279 await SentMessage(user, "开始解压缩文件。。。");
280 #if DEBUG
281 Debug.WriteLine("开始解压缩文件。。。");
282 #endif
283 fastZip.ExtractZip(zipFile, zipExtDir, "");
284 #if DEBUG
285 Debug.WriteLine("解压缩文件成功。。。");
286 #endif
287 await SentMessage(user, "解压缩文件成功,开始校验。。。");
288 //发送解压成功并开始校验文件信息
289 var zipFiles = Directory.GetFiles(zipExtDir, "*.jpg", SearchOption.AllDirectories);
290 for (var i = 0; i < zipFiles.Length; i++)
291 {
292 var file = zipFiles[i];
293 var i1 = i + 1;
294 await Task.Delay(100);//模拟文件处理需要100毫秒
295 //发送进度 i/length
296 await SentMessage(user, $"校验进度==>{i1}/{zipFiles.Length}", "400");
297 #if DEBUG
298 Debug.WriteLine($"当前进度:{i1},总数:{zipFiles.Length}");
299 #endif
300 }
301 await SentMessage(user, "校验完成", "200");
302 }
303 catch (Exception exception)
304 {
305 //发送解压缩失败信息
306 await SentMessage(user, $"解压缩文件失败:{exception}", "500");
307 #if DEBUG
308 Debug.WriteLine($"解压缩文件失败:{exception}");
309 #endif
310 }
311 }, TaskCreationOptions.LongRunning);
312 }
313
314 #endregion
315
316 #region 消息推送前台
317
318 private async Task SentMessage(string user, string content, string code = "300")
319 {
320
321 await _hubContext.Clients.Client(user).UploadInfoMessage(new ClientMessageModel
322 {
323 UserId = user,
324 GroupName = "upload",
325 Context = content,
326 Code = code
327 });
328 }
329
330 #endregion
331 }
332 }
未能完善的地方:
1、上传几百兆或更大的文件,webuploader计算md5时间太长;
2、后台处理错误的时候,前台接收消息后没能出现关闭按钮;
3、分块传输时文件断点续传没有具体实现(理论上是没问题的)
参考文章:
https://www.cnblogs.com/wdw984/p/11725118.html
http://fex.baidu.com/webuploader/
如此文章对你有帮助,请点个推荐吧。谢谢!
手机扫一扫
移动阅读更方便
你可能感兴趣的文章