Recently I was asked a simple code to improve the performance, and at first I couldn’t see it but as you know your brain starts working when you aren’t thinking of the problem anymore. (It happen to me when I was driving)
Anyway the problem on hand was there is a for loop and when there are 10000 clients it performs bad.
Something along the line of.
|
1 2 3 4 5 6 7 8 9 10 11 12 |
List<Client> _client; public void SendMessage(int id) { foreach(var c in _client) { if(c.Id == id) { c.SendMessage(); break; } } |
So I went and wrote some code to see the perf using Linq and at the end using a Dictionary rather, since a dictionary is way faster in lookup, O(1) in this case.
Here is the code using a List and Linq together
|
1 2 3 4 |
foreach (var client in _clients.Where(x => x.Id == id)) { client.SendMessage(); } |
and if we changed the datatype to Dictionary this was the code used
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
//using tryget if (_clients.TryGetValue(id, out c)) { c.SendMessage(); } //using linq var c = _clients.FirstOrDefault(x => x.Key == id).Value; if (c != null) { c.SendMessage(); } //using just an index Client c = _clients[id]; c.SendMessage(); |
Here is the end result of using it on an i7 8G of ram machine.
And the results show that using an index as a dictionary is definitely faster, but one also has to look at the number of data.
Where there are only few data entry using linq is quite insignificant in performance but where there are millions of data, it really shows that it slows down.
Here is the rest of the code.
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 |
using System; using System.Collections.Generic; using System.Linq; namespace PerfMeasureApp { class Program { static void Main(string[] args) { var queue = new MessageQueue {Clients = GenerateListClients()}; var fastQueue = new MessageQueueSmart {Clients = GenerateDictClients()}; queue.DoStuff(1); Console.WriteLine("********************************************"); queue.DoStuffLinq(1); Console.WriteLine("********************************************"); fastQueue.DoStuff(9999999); Console.WriteLine("********************************************"); fastQueue.DoStuffLinq(9999999); Console.WriteLine("********************************************"); fastQueue.DoStuffIndex(9999999); Console.WriteLine("********************************************"); Console.Read(); } // Define other methods and classes here public static List<Client> GenerateListClients() { var list = new List<Client>(); for (int i = 0; i < 10000000; i++) { if (i == 9999999) list.Add(new Client(1)); else list.Add(new Client(0)); } return list; } public static Dictionary<int, Client> GenerateDictClients() { var dict = new Dictionary<int, Client>(); for (int i = 0; i < 10000000; i++) { dict.Add(i, new Client(0)); } return dict; } } public class MessageQueueSmart { private IDictionary<int, Client> _clients; public IDictionary<int, Client> Clients { set { _clients = value; } } public void DoStuff(int id) { Console.WriteLine("MessageQueueSmart DoStuffTryGet"); System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch(); stopwatch.Start(); Client c; if (_clients.TryGetValue(id, out c)) { c.SendMessage(); } stopwatch.Stop(); Console.WriteLine("Time elapsed: {0}", stopwatch.Elapsed); } public void DoStuffLinq(int id) { Console.WriteLine("MessageQueueSmart DoStuffLinq"); System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch(); stopwatch.Start(); var c = _clients.FirstOrDefault(x => x.Key == id).Value; if (c != null) { c.SendMessage(); } stopwatch.Stop(); Console.WriteLine("Time elapsed: {0}", stopwatch.Elapsed); } public void DoStuffIndex(int id) { Console.WriteLine("MessageQueueSmart DoStuffIndex"); System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch(); stopwatch.Start(); Client c = _clients[id]; c.SendMessage(); stopwatch.Stop(); Console.WriteLine("Time elapsed: {0}", stopwatch.Elapsed); } } public class MessageQueue { private List<Client> _clients; public List<Client> Clients { set { _clients = value; } } public void DoStuff(int id) { Console.WriteLine("MessageQueue DoStuff"); System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch(); stopwatch.Start(); foreach (var client in _clients) { if (client.Id == id) { client.SendMessage(); break; } } stopwatch.Stop(); Console.WriteLine("Time elapsed: {0}", stopwatch.Elapsed); } public void DoStuffLinq(int id) { Console.WriteLine("MessageQueue DoStuffLinq"); System.Diagnostics.Stopwatch stopwatch = new System.Diagnostics.Stopwatch(); stopwatch.Start(); foreach (var client in _clients.Where(x => x.Id == id)) { client.SendMessage(); } stopwatch.Stop(); Console.WriteLine("Time elapsed: {0}", stopwatch.Elapsed); } } public class Client { public Client(int id) { Id = id; } public int Id { get; set; } public void SendMessage() { Console.WriteLine("SendMessage called " + Id); } } } |
I don’t think Linq is an issue when processing millions of records if the records are being processed over a period of hours or days or years. It depends on long you need it to take. Linq makes code easy to read, maintain and write. Sometimes it’s ok to make that trade off. Only avoid Linq when it’s not fast enough.
I definitely agree with you, and I am a strong believer/user of Linq. I dont think one should avoid it but should know the trade-offs of using any technologies 🙂
Thanks for the comment.
Thanks for this. I have a dataset that can contain upwards to 10k items at a time, and I needed a good way to search without LinQ. It’s amazing how much you forget from college Data Structs course after a few years.