Network Working Group K. PoduriRequest for Comments: 2415 K. NicholsCategory: Informational Bay Networks September 1998 Simulation Studies of Increased Initial TCP Window SizeStatus of this Memo This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.Copyright Notice Copyright (C) The Internet Society (1998). All Rights Reserved.Abstract An increase in the permissible initial window size of a TCP connection, from one segment to three or four segments, has been under discussion in the tcp-impl working group. This document covers some simulation studies of the effects of increasing the initial window size of TCP. Both long-lived TCP connections (file transfers) and short-lived web-browsing style connections were modeled. The simulations were performed using the publicly available ns-2 simulator and our custom models and files are also available.1. IntrodUCtion We present results from a set of simulations with increased TCP initial window (IW). The main objectives were to eXPlore the conditions under which the larger IW was a 'win' and to determine the effects, if any, the larger IW might have on other traffic flows using an IW of one segment. This study was inspired by discussions at the Munich IETF tcp-impl and tcp-sat meetings. A proposal to increase the IW size to about 4K bytes (4380 bytes in the case of 1460 byte segments) was discussed. Concerns about both the utility of the increase and its effect on other traffic were raised. Some studies were presented showing the positive effects of increased IW on individual connections, but no studies were shown with a wide variety of simultaneous traffic flows. It appeared that some of the questions being raised could be addressed in an ns-2 simulation. Early results from our simulations were previously posted to the tcp-impl mailing list and presented at the tcp-impl WG meeting at the December 1997 IETF.2. Model and Assumptions We simulated a network topology with a bottleneck link as shown: 10Mb, 10Mb, (all 4 links) (all 4 links) C n2_________ ______ n6 S l n3_________ /______ n7 e i 1.5Mb, 50ms // r e n0 ------------------------ n1 v n n4__________// _____ n8 e t n5__________/ ______ n9 r s s URLs --> <--- FTP & Web data File downloading and web-browsing clients are attached to the nodes (n2-n5) on the left-hand side. These clients are served by the FTP and Web servers attached to the nodes (n6-n9) on the right-hand side. The links to and from those nodes are at 10 Mbps. The bottleneck link is between n1 and n0. All links are bi-directional, but only ACKs, SYNs, FINs, and URLs are flowing from left to right. Some simulations were also performed with data traffic flowing from right to left simultaneously, but it had no effect on the results. In the simulations we assumed that all ftps transferred 1-MB files and that all web pages had exactly three embedded URLs. The web clients are browsing quite aggressively, requesting a new page after a random delay uniformly distributed between 1 and 5 seconds. This is not meant to realistically model a single user's web-browsing pattern, but to create a reasonably heavy traffic load whose individual tcp connections accurately reflect real web traffic. Some discussion of these models as used in earlier studies is available in references [3] and [4]. The maximum tcp window was set to 11 packets, maximum packet (or segment) size to 1460 bytes, and buffer sizes were set at 25 packets. (The ns-2 TCPs require setting window sizes and buffer sizes in number of packets. In our tcp-full code some of the internal parameters have been set to be byte-oriented, but external values must still be set in number of packets.) In our simulations, we varied the number of data segments sent into a new TCP connection (or initial window) from one to four, keeping all segments at 1460 bytes. A dropped packet causes a restart window of one segment to be used, just as in current practice. For ns-2 users: The tcp-full code was modified to use an 'application' class and three application client-server pairs were written: a simple file transfer (ftp), a model of http1.0 style web connection and a very rough model of http1.1 style web connection. The required files and scripts for these simulations are available under the contributed code section on the ns-simulator web page at the sites ftp://ftp.ee.lbl.gov/IW.{tar, tar.Z} or http://www- nrg.ee.lbl.gov/floyd/tcp_init_win.Html. Simulations were run with 8, 16, 32 web clients and a number of ftp clients ranging from 0 to 3. The IW was varied from 1 to 4, though the 4-packet case lies beyond what is currently recommended. The figures of merit used were goodput, the median page delay seen by the web clients and the median file transfer delay seen by the ftp clients. The simulated run time was rather large, 360 seconds, to ensure an adequate sample. (Median values remained the same for simulations with larger run times and can be considered stable)3. Results In our simulations, we varied the number of file transfer clients in order to change the congestion of the link. Recall that our ftp clients continuously request 1 Mbyte transfers, so the link utilization is over 90% when even a single ftp client is present. When three file transfer clients are running simultaneously, the resultant congestion is somewhat pathological, making the values recorded stable. Though all connections use the same initial window, the effect of increasing the IW on a 1 Mbyte file transfer is not detectable, thus we focus on the web browsing connections. (In the tables, we use 'webs' to indicate number of web clients and 'ftps' to indicate the number of file transfer clients attached.) Table 1 shows the median delays experienced by the web transfers with an increase in the TCP IW. There is clearly an improvement in transfer delays for the web connections with increase in the IW, in many cases on the order of 30%. The steepness of the performance improvement going from an IW of 1 to an IW of 2 is mainly due to the distribution of files fetched by each URL (see references [1] and [2]); the median size of both primary and in-line URLs fits completely into two packets. If file distributions change, the shape of this curve may also change. Table 1. Median web page delay #Webs #FTPs IW=1 IW=2 IW=3 IW=4 (s) (% decrease) ---------------------------------------------- 8 0 0.56 14.3 17.9 16.1 8 1 1.06 18.9 25.5 32.1 8 2 1.18 16.1 17.1 28.9 8 3 1.26 11.9 19.0 27.0 16 0 0.64 11.0 15.6 18.8 16 1 1.04 17.3 24.0 35.6 16 2 1.22 17.2 20.5 25.4 16 3 1.31 10.7 21.4 22.1 32 0 0.92 17.6 28.6 21.0 32 1 1.19 19.6 25.0 26.1 32 2 1.43 23.8 35.0 33.6 32 3 1.56 19.2 29.5 33.3 Table 2 shows the bottleneck link utilization and packet drop percentage of the same experiment. Packet drop rates did increase with IW, but in all cases except that of the single most pathological overload, the increase in drop percentage was less than 1%. A decrease in packet drop percentage is observed in some overloaded situations, specifically when ftp transfers consumed most of the link bandwidth and a large number of web transfers shared the remaining bandwidth of the link. In this case, the web transfers experience severe packet loss and some of the IW=4 web clients suffer multiple packet losses from the same window, resulting in longer recovery times than when there is a single packet loss in a window. During the recovery time, the connections are inactive which alleviates congestion and thus results in a decrease in the packet drop percentage. It should be noted that such observations were made only in extremely overloaded scenarios.Table 2. Link utilization and packet drop rates Percentage Link Utilization Packet drop rate#Webs #FTPs IW=1 IW=2 IW=3 IW=4 IW=1 IW=2 IW=3 IW=4----------------------------------------------------------------------- 8 0 34 37 38 39 0.0 0.0 0.0 0.0 8 1 95 92 93 92 0.6 1.2 1.4 1.3 8 2 98 97 97 96 1.8 2.3 2.3 2.7 8 3 98 98 98 98 2.6 3.0 3.5 3.5----------------------------------------------------------------------- 16 0 67 69 69 67 0.1 0.5 0.8 1.0 16 1 96 95 93 92 2.1 2.6 2.9 2.9 16 2 98 98 97 96 3.5 3.6 4.2 4.5 16 3 99 99 98 98 4.5 4.7 5.2 4.9----------------------------------------------------------------------- 32 0 92 87 85 84 0.1 0.5 0.8 1.0 32 1 98 97 96 96 2.1 2.6 2.9 2.9 32 2 99 99 98 98 3.5 3.6 4.2 4.5 32 3 100 99 99 98 9.3 8.4 7.7 7.6 To get a more complete picture of performance, we computed the network power, goodput divided by median delay (in Mbytes/ms), and plotted it against IW for all scenarios. (Each scenario is uniquely identified by its number of webs and number of file transfers.) We plot these values in Figure 1 (in the pdf version), illustrating a general advantage to increasing IW. When a large number of web clients is combined with ftps, particularly multiple ftps, pathological cases result from the extreme congestion. In these cases, there appears to be no particular trend to the results of increasing the IW, in fact simulation results are not particularly stable. To get a clearer picture of what is happening across all the tested scenarios, we normalized the network power values for the non- pathological scenario by the network power for that scenario at IW of one. These results are plotted in Figure 2. As IW is increased from one to four, network power increased by at least 15%, even in a congested scenario dominated by bulk transfer traffic. In simulations where web traffic has a dominant share of the available bandwidth, the increase in network power was up to 60%. The increase in network power at higher initial window sizes is due to an increase in throughput and a decrease in the delay. Since the (slightly) increased drop rates were accompanied by better performance, drop rate is clearly not an indicator of user level performance. The gains in performance seen by the web clients need to be balanced against the performance the file transfers are seeing. We computed ftp network power and show this in Table 3. It appears that the improvement in network power seen by the web connections has negligible effect on the concurrent file transfers. It can be observed from the table that there is a small variation in the network power of file transfers with an increase in the size of IW but no particular trend can be seen. It can be concluded that the network power of file transfers essentially remained the same. However, it should be noted that a larger IW does allow web transfers to gain slightly more bandwidth than with a smaller IW. This could mean fewer bytes transferred for FTP applications or a slight decrease in network power as computed by us. Table 3. Network power of file transfers with an increase in the TCP IW size #Webs #FTPs IW=1 IW=2 IW=3 IW=4 -------------------------------------------- 8 1 4.7 4.2 4.2 4.2 8 2 3.0 2.8 3.0 2.8 8 3 2.2 2.2 2.2 2.2 16 1 2.3 2.4 2.4 2.5 16 2 1.8 2.0 1.8 1.9 16 3 1.4 1.6 1.5 1.7 32 1 0.7 0.9 1.3 0.9 32 2 0.8 1.0 1.3 1.1 32 3 0.7 1.0 1.2 1.0 The above simulations all used http1.0 style web connections, thus, a natural question is to ask how results are affected by migration to http1.1. A rough model of this behavior was simulated by using one connection to send all of the information from both the primary URL and the three embedded, or in-line, URLs. Since the transfer size is now made up of four web files, the steep improvement in performance between an IW of 1 and an IW of two, noted in the previous results, has been smoothed. Results are shown in Tables 4 & 5 and Figs. 3 & 4. Occasionally an increase in IW from 3 to 4 decreases the network power owing to a non-increase or a slight decrease in the throughput. TCP connections opening up with a higher window size into a very congested network might experience some packet drops and consequently a slight decrease in the throughput. This indicates that increase of the initial window sizes to further higher values (>4) may not always result in a favorable network performance. This can be seen clearly in Figure 4 where the network power shows a decrease for the two highly congested cases. Table 4. Median web page delay for http1.1 #Webs #FTPs IW=1 IW=2 IW=3 IW=4 (s) (% decrease) ---------------------------------------------- 8 0 0.47 14.9 19.1 21.3 8 1 0.84 17.9 19.0 25.0 8 2 0.99 11.5 17.3 23.0 8 3 1.04 12.1 20.2 28.3 16 0 0.54 07.4 14.8 20.4 16 1 0.89 14.6 21.3 27.0 16 2 1.02 14.7 19.6 25.5 16 3 1.11 09.0 17.0 18.9 32 0 0.94 16.0 29.8 36.2 32 1 1.23 12.2 28.5 21.1 32 2 1.39 06.5 13.7 12.2 32 3 1.46 04.0 11.0 15.0 Table 5. Network power of file transfers with an increase in the TCP IW size #Webs #FTPs IW=1 IW=2 IW=3 IW=4 -------------------------------------------- 8 1 4.2 4.2 4.2 3.7 8 2 2.7 2.5 2.6 2.3 8 3 2.1 1.9 2.0 2.0 16 1 1.8 1.8 1.5 1.4 16 2 1.5 1.2 1.1 1.5 16 3 1.0 1.0 1.0 1.0 32 1 0.3 0.3 0.5 0.3 32 2 0.4 0.3 0.4 0.4 32 3 0.4 0.3 0.4 0.5 For further insight, we returned to the http1.0 model and mixed some web-browsing connections with IWs of one with those using IWs of three. In this experiment, we first simulated a total of 16 web- browsing connections, all using IW of one. Then the clients were split into two groups of 8 each, one of which uses IW=1 and the other used IW=3. We repeated the simulations for a total of 32 and 64 web-browsing clients, splitting those into groups of 16 and 32 respectively. Table 6 shows these results. We report the goodput (in Mbytes), the web page delays (in milli seconds), the percent utilization of the link and the percent of packets dropped.Table 6. Results for half-and-half scenarioMedian Page Delays and Goodput (MB) Link Utilization (%) & Drops (%)#Webs IW=1 IW=3 IW=1 IW=3 G.put dly G.put dly L.util Drops L.util Drops-------------------------------------------------------------------16 35.5 0.64 36.4 0.54 67 0.1 69 0.78/8 16.9 0.67 18.9 0.52 68 0.5 -------------------------------------------------------------------32 48.9 0.91 44.7 0.68 92 3.5 85 4.316/16 22.8 0.94 22.9 0.71 89 4.6 --------------------------------------------------------------------64 51.9 1.50 47.6 0.86 98 13.0 91 8.632/32 29.0 1.40 22.0 1.20 98 12.0 Unsurprisingly, the non-split experiments are consistent with our earlier results, clients with IW=3 outperform clients with IW=1. The results of running the 8/8 and 16/16 splits show that running a mixture of IW=3 and IW=1 has no negative effect on the IW=1 conversations, while IW=3 conversations maintain their performance. However, the 32/32 split shows that web-browsing connections with IW=3 are adversely affected. We believe this is due to the pathological dynamics of this extremely congested scenario. Since embedded URLs open their connections simultaneously, very large number of TCP connections are arriving at the bottleneck link resulting in multiple packet losses for the IW=3 conversations. The myriad problems of this simultaneous opening strategy is, of course, part of the motivation for the development of http1.1.4. Discussion The indications from these results are that increasing the initial window size to 3 packets (or 4380 bytes) helps to improve perceived performance. Many further variations on these simulation scenarios are possible and we've made our simulation models and scripts available in order to facilitate others' experiments. We also used the RED queue management included with ns-2 to perform some other simulation studies. We have not reported on those results here since we don't consider the studies complete. We found that by adding RED to the bottleneck link, we achieved similar performance gains (with an IW of 1) to those we found with increased IWs without RED. Others may wish to investigate this further. Although the simulation sets were run for a T1 link, several scenarios with varying levels of congestion and varying number of web and ftp clients were analyzed. It is reasonable to expect that the results would scale for links with higher bandwidth. However, interested readers could investigate this ASPect further. We also used the RED queue management included with ns-2 to perform some other simulation studies. We have not reported on those results here since we don't consider the studies complete. We found that by adding RED to the bottleneck link, we achieved similar performance gains (with an IW of 1) to those we found with increased IWs without RED. Others may wish to investigate this further.5. References [1] B. Mah, 'An Empirical Model of HTTP Network Traffic', Proceedings of INFOCOM '97, Kobe, Japan, April 7-11, 1997. [2] C.R. Cunha, A. Bestavros, M.E. Crovella, 'Characteristics of WWW Client-based Traces', Boston University Computer Science Technical Report BU-CS-95-010, July 18, 1995. [3] K.M. Nichols and M. Laubach, 'Tiers of Service for Data Access in a HFC Architecture', Proceedings of SCTE Convergence Conference, January, 1997. [4] K.M. Nichols, 'Improving Network Simulation with Feedback', available from knichols@baynetworks.com6. Acknowledgements This work benefited from discussions with and comments from Van Jacobson.7. Security Considerations This document discusses a simulation study of the effects of a proposed change to TCP. Consequently, there are no security considerations directly related to the document. There are also no known security considerations associated with the proposed change.8. Authors' Addresses Kedarnath Poduri Bay Networks 4401 Great America Parkway SC01-04 Santa Clara, CA 95052-8185 Phone: +1-408-495-2463 Fax: +1-408-495-1299 EMail: kpoduri@Baynetworks.com Kathleen Nichols Bay Networks 4401 Great America Parkway SC01-04 Santa Clara, CA 95052-8185 EMail: knichols@baynetworks.comFull Copyright Statement Copyright (C) The Internet Society (1998). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an 'AS IS' basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.