VMware PXE Limitations
Update 12/12/2014: While the information below is valid that VMware does not appear to support RamDiskTFTPBlockSize above 1432 due to no support to deal with IP Fragmentation. However, it has been found that it is much better to adjust RamDiskTFTPWindowSize instead of RamDiskTFTPBlockSize to speed up TFTP (reduces amount of ACKs required and does not cause IP Fragmentation). VMware and other vendors all appear to handle this scenario perfectly. Modifying the WindowSize is easy to do from most PXE providers, including WDS (modify BCD). There is an issue changing this setting in Configuration Manager due to dynamic BCD that ConfigMgr manages itself with no native way to modify the setting (see the comments section for how to workaround this issue – thanks Patrick!). More info throughout the comments section.
Update 3/18/2016: Microsoft has released an update (currently in preview) that will now allow modifying both the WindowSize and the BlockSize without a hacked DLL. Info here: https://technet.microsoft.com/en-us/library/mt595861.aspx#BKMK_RC1603. This update is in 1603 Technical Preview and should also be apart of the next Current Branch release of Configuration Manager.
Update 7/22/2016: Microsoft has released a new Current Branch of SCCM, version 1606. I have confirmed that Current Branch version 1606 contains these modifications that were in version 1603 Preview. I have also tested the WindowSize/BlockSize registry keys and they are working great.
Example:
Key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP
Type: DWORD
Name: RamDiskTFTPWindowSize
Value: 6
Note: After making changes, make sure to restart the Windows Deployment Service.
We recently ran into some issues, where on various hardware settings, PXE (on physical hardware from various vendors) was very slow (10 minutes to copy 150 MB). After doing some research we found this issue could be easily rectified by changing the RamDiskTFTPBlockSize to a large value (such as 16384). We use SCCM 2012 R2 for our PXE server, but this setting can be changed directly on WDS, or SCCM 2007 as well. I also believe the same change can be made on other third party PXE providers as well, I just haven’t tested those. Here is a good blog on how to change your RamDiskTFTPBlockSize for various environments: http://windowsdeployments.net/how-to-speed-up-pxe-boot-in-wds-and-sccm/
We found as soon as we made the RamDiskTFTPBlockSize change, PXE was lighting fast everywhere. Examples from our testing are: Microsoft Hyper-V VM, Dell, HP, Wyse, 10ZiG. Everything was great, except for VMware, where PXE boot now fails.
We tried using the following settings for the block size: 2048, 4096, 8192, 16384 and they all fail on VMware with error code 0xC0000001. These values worked fine on all other systems. When using the E1000 NIC, the info on the error states: “A required device isn’t connected or can’t be accessed”. When using VMXNet3, the info on the error states “File: \Windows\System32\boot\winload.efi; Info: The application or operating system couldn’t be loaded because a required file is missing or contains error”. The failures occur whether the firmware is BIOS of UEFI (error message just slightly deviates). Based on the errors, it was clear that VMware was failing the TFTP transfer and leaving corrupt data.
To try to find the source of the problem, we ran packet captures in VMware to see where the breakdown was happening. We also sent these off to VMware Support for analysis (getting VMware support involved was actually a very painful process; which took over a month to get an answer back, despite calling and emailing support frequently with no response). Once VMware finally got back to us the answer unfortunately was not very positive. VMware states: 1) IP reassembly is not implemented in PXE code and 2) ESXi PXE does not support jumbo frames.
This basically means that VMware cannot support RamDiskTFTPBlockSize being any bigger than 1432. This is a pretty big limitation since this affects our ability to reasonably support our physical machines and VMware machines at the same time. It also seems bad since VMware is the only vendor I can find that has this problem. It wouldn’t be a huge problem if at least supported 8192, but not even 2048 works.
If anyone else has this problem and has found a good workaround, please leave notes in the comments below. Hopefully this information will save some troubleshooting time.
I’m experiencing a very similar and probably related problem.
We are using Windows Server 2012 R2 with vmxnet3, SCCM 2012 R2 with CU3 and booting from PXE on site servers.
Most locations are taking 10 minutes but some are taking up to 40 minutes booting from a local server.
I also have tried to set the RamDiskTFTPBlockSize as well as WindowSize to all possible values.
Differently from you I can notice the difference when I set those values but for worse not for better!
Looking for possible solutions.
Cheers
Interesting. Are you ever able to successfully when PXE booting in VMware with RamDiskTFTPBlockSize larger than 1432; or does it always come back failed (corrupt?) – in addition to being worse in time?
Yes I can successfully PXE boot. I just don’t create the RamDiskTFTPBlockSize so it should use the default value of 1024.
I have found out something really interesting. If I use the build-in SCCM boot images instead of the MDT boot images it goes much faster even though they have the same size…
Ok, I assume you can no longer successfully PXE boot at all (like me) if you don’t allow RamDiskTFTPBlockSize to be default value of 1024? (ie. create and set value to 2048, 4096, etc.)
Interesting that SCCM boot images would be different than MDT, I wouldn’t think they would be since PXE is just using TFTP to move raw data. I don’t have MDT setup in my environment, so I can’t compare.
I can still PXE with some different values but it gets even slower.
It’s weird indeed the difference between times…
Are your DPs setup as standalone site systems or are they site servers?
Interesting, when I use other values it comes back at the same speed just corrupt and fails. My DPs at one site is a site system, my DP at another site is standalone, same results on both.
What VMware answered is right.
Most of the PXE clients out there do not support jumbo frames, IP fragmentation and the like.
TFTP is originally a lock-step protocol that for every data block transferred the DataSender stops for the ACK coming from the DataReceiver.
If the round trip delay is big, regular TFTP transfers get negatively affected. The TFTP protocol originally ruled a block size of only 512 bytes, next the negotiated “blocksize” option extension allowed to set block sizes of any size but limited to 1468 bytes in Ethernet environments with MTU=1500.
When you guys “touch” RamDiskTFTPBlockSize is to set a bigger block size in order to need fewer ACKs “but” you also get the unwanted IP fragmentation side effect. Not good.
To really solve this problem you guys have to deal with the “windowed” TFTP concept; the idea is the DataSender sends “windowsize” consecutive data blocks before to stop for a “single” ACK. This way we are able to avoid IP fragmentation (blocksize <= 1468) and we need to wait for fewer ACKs what implies faster transfers.
Microsoft implements 2 schemas with fixed and variable "windowsize" and that can be set at bcd level.
Best,
Patrick
Patrick, sorry for the slow reply over the holiday. Thanks for the great information and insight. We actually did try making adjustments to the window sizes without any positive improvements. We have the setting “Enable Variable Window Extension” turned on in Windows TFTP (variable window size) and it doesn’t help our slow clients. We also tried hard coding a fixed window sizes, and also saw no changes.
The reason we moved to changing RamRiskTFTPBlockSizes is because it was the only setting that made a positive difference on the slow clients. It seems, that while IP fragmentation is not ideal, most PXE clients (HP, Dell, Wyse, ASUS, Microsoft, etc.) seem to handle it fine, just not for VMware.
Any ideas why changing the Window Sizes didn’t help our slow PXE clients? In VMware the speed seems to be the same no matter what either setting is (but gets corrupt when IP Fragmentation occurs).
At this point you should use a sniffer (Wireshark) running at your server and see what’s really going on at packet level.
I would not rely on RiskTFTPBlockSizes; I’d just try to analyze what happens when using windowsize 4 or 8 . The wireshark traffic capture will surely help.
Some idea that comes to my mind is that when you rely on big RiskTFTPBlockSizes the number of data blocks get reduced but when you rely on a windowsize strategy the number of data-packets remains the same only the number of ACKs get reduced, that tells me the delay is produced while transferring the data-packets; that could be the sign of some contrack/firewall layer adding an extra delay for every packet transferred. Please take a look at RFC1350 and see that TFTP traffic requires UDP port 69 open but also a random port used by the server where the actual data-transfer takes place (similar to FTP). If the traffic on this port is for some reason delayed by any intermediate security/track control layer you will see exactly the problem that you describe here. For this reason you must be sure i.e. there’s no any NAT service between your TFTP client and Server. I hope you get the idea.
Best,
Patrick
Patrick, sorry for the delay to complete your suggestion. I ran some Wire Shark captures and don’t really see a clear cause of any problems besides that it is just slow. Below is a relevant snippet of the capture.
Essentially, the server just repeats Data Packet followed by Acknowledgement until it gets to 65535 and then that counter starts over until finally finishes. This test took over 6 minutes to transfer a file under 200 MB. There is one failure at the top, but it immediately retries and works. Any thoughts?
No. Time Source Destiniation Protocol Length Info
1893 41.072709000 CLIENT SERVER UDP 389 Source port: 4011 Destination port: 4011
1896 41.225051000 SERVER CLIENT UDP 472 Source port: 4011 Destination port: 4011
1917 42.117013000 CLIENT SERVER TFTP 95 Read Request, File: smsboot\x64\wdsmgfw.efi, Transfer type: octet, tsize\000=0\000, blksize\000=1468\000
1918 42.117542000 SERVER CLIENT TFTP 71 Option Acknowledgement, blksize\000=1468\000, tsize\000=1459552\000
1919 42.117958000 CLIENT SERVER TFTP 72 Error Code, Code: Option negotiation failed, Message: User aborted the transfer
1920 42.157829000 CLIENT SERVER TFTP 87 Read Request, File: smsboot\x64\wdsmgfw.efi, Transfer type: octet, blksize\000=1468\000
1921 42.158280000 SERVER CLIENT TFTP 57 Option Acknowledgement, blksize\000=1468\000
1922 42.158605000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 0
1923 42.158970000 SERVER CLIENT TFTP 1514 Data Packet, Block: 1
1924 42.159401000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 1
1925 42.159459000 SERVER CLIENT TFTP 1514 Data Packet, Block: 2
1926 42.159902000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 2
1927 42.159939000 SERVER CLIENT TFTP 1514 Data Packet, Block: 3
1928 42.160329000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 3
…
297793 507.978765000 SERVER CLIENT TFTP 1478 Data Packet, Block: 65535
297794 507.981821000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 65535
297795 507.981922000 SERVER CLIENT TFTP 1478 Data Packet, Block: 0
297795 507.981922000 SERVER CLIENT TFTP 1478 Data Packet, Block: 0
…
349347 613.783380000 SERVER CLIENT TFTP 558 Data Packet, Block: 60
349348 613.783691000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 60
349349 613.783750000 SERVER CLIENT TFTP 86 Data Packet, Block: 61 (last)
349350 613.784081000 CLIENT SERVER TFTP 60 Acknowledgement, Block: 61
what you have done is just a test of a plain RFC1350 transfer without windowsize. To fully understand the capture I’d should see the timing of the packets that is missing in your posting. upload that capture somewhere and I can take a look at it.
try to run a capture with windosize =4 or 8 just to compare.
I usually transfer ~200 MB wim files with winsize=8 in about 30/40 secs
Patrick, I have uploaded the full capture here: (http://1drv.ms/138j59v), let me know once you have looked at it
Your reply actually makes a lot more sense after some further research. Since we are using System Center Configuration Manager 2012 R2 on top of our WDS server, apparently there are some differences with the TFTP client used (https://social.technet.microsoft.com/Forums/en-US/ae6121d2-18ea-40d4-b662-a2a2ec7d1f18/slow-pxe-boot-tftp-windowing-issue?forum=configmanagerosd). There is supposed to be a DWORD registry key under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP called RamDiskTFTPWindowSize, that I can set to either 4 or 8. Apparently this key does not work. The RamDiskTFTPBlockSize DWORD at this same location does work, but leads to issues as we know.
One thing about the Configuration Manager implementation is that default.bcd does not exist, so I can’t just force it manually at BCD level. Configuration Manager internally creates a temporary BCD file for each and every PXE request (I can view these temp BCD files and can confirm when RamDiskTFTPBlockSize changes, but RamDiskTFTPWindowSize is always missing).
Therefore, all this time, since I had this registry key set, I thought I was using a window size of 8, until we ran the capture, which tells us we are not using a WindowSize. Best I can tell from this information, there is no way I can use a WindowSize here with Configuration Manager?
Also, still not sure why some hardware is slow and some is not when we are not using WindowSizes (RealTek NICs seem to be the slowest and unusable, but other NICs like VMware are acceptable without WindowSizes (about 1-2 minutes)?
Thanks!!
I studied your capture; taking randomly the round-trip delay it looks consistent around 3.2 mS with a payload of 1432 bytes then considering the transfer (boot.wim) takes 527 seconds from Wireshark we have; 527 s / 0.0032 s * 1432 Bytes = 224 MB sure consistent with the size of a Boot.wim today. Then your TFTP is OK, it’s consistent, but slow… You asked why? why this only happens on some devices? well there are cards that run at 1GB and cards that cannot negotiate that speed, plus there are some with faulty or outdated PXE firmware; that’s it.
I read the link and it seems SCCM is not adding the windowing parameter to the BCD; the guy that says that the SCCM client doesn’t deal with windowed TFTP is mistaken. It is the bootmgr the one that deals with those TFTP options from the client and the SCCM bootmgr has the windowed TFTP options.
Your goal now is making SCCM honoring the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP\RamdiskTFTPWindowSize that should trigger the adding of the corresponding parameter to the BCD.
If you want to know in advance if a windowed TFTP schema will fix your problem take a look at Serva; http://vercot.com/~serva/an/WindowsPXE1.html
You will be able to quickly test a Windows install with different values of windowsize; the difference is really big.
if you want to go a bit deeper on this try to see which process (name of the dll) is touching RamDiskTFTPBlockSize when creating the BCD; do it with Rusinovich’s procmon or something like that.
Patrick, thanks, that all makes a lot of sense. I figured the root issue was likely hardware related (bad PXE firmware, etc.); unfortunately this issue is relatively wide spread across cards and hard to just avoid them. It does seem like RamdiskTFTPWindowSize will solve the issue, if can get that feature working with Configuration Manager. Have you ever seen it work with ConfigMgr? I reached out to Microsoft to see if any feedback from them on why the registry key does not work, will see what I hear back from them. I also ran procmon to see what is creating the BCD, looks like its a combination of svchost.exe and wmiprvse.exe; which doesn’t help much…
OK, in SCCM the BCD is created by \BIN\I386\smspxe.dll or \BIN\X64smspxe.dll. I have studied that dll and it is true; while it parses RamDiskTFTPBlockSize it never parses RamdiskTFTPWindowSize. While I can clearly see the code that injects RamDiskTFTPBlockSize in the BCD the code that injects RamdiskTFTPWindowSize is obviously missing.
I have no a running SCCM handy now then if you have a minute just try this.
1) back-up your BIN\X64\smspxe.dll
2) open it with a hex editor like Hex Workshop
3) search the “unicode” string “((ULONG)(0x000007)” w/o quotes, it should appear only once.
4) replace the char “7” with an “8” and save the change.
5) Set the RamDiskTFTPBlockSize registry key to 8
6) Boot a client.
What you just did was a dirty patch of the dll that will read RamDiskTFTPBlockSize from the registry but it will inject its value in the BCD as the corresponding RamdiskTFTPWindowSize parameter.
Now the client will receive in the BCD RamdiskTFTPWindowSize=8 and it’ll take a default for the missing RamDiskTFTPBlockSize value.
If anything goes wrong just replace the patched dll with the back-up and delete the RamDiskTFTPBlockSize registry key. No big deal
Let me know if it works.
Best,
Patrick
a little change..
the patch described before only changes the logging of the operation
for really patching the “action” try these steps instead
3) search the hexadecimal sequence 07000035, it should appear only once.
4) replace the byte 07 with 08 and save the change.
Best,
Patrick
Ok, I was just about to reply stating it didn’t work 🙂 I confirmed by looking at file locks, it appears in my environment (x64 OS) that the x64 version of smspxe.dll is used. Should both changes be made, or just your second change? I’ll test it now. Thanks.
Patrick, also, I just searched for 07000035 and actually got two instances found. Which one needs to be changed? When viewing the hex, should be in Unicode or Legacy ASCII?
I have the version 5.0.7804.1000 and it has only 1 instance.
(not installed)
in your case (without seeing the code) I would try changing one, the other one and both instances…
My version is 5.0.7958.1401. In case you would like to look at it; I put a copy of the DLL here: http://1drv.ms/1zHHLlb
I will start testing each value
in your case is the first instance; the sequence to find is BA07000035
This works! (I had just finishing trying both values and can confirm this value is the one that works as well). This makes massive improvements in the TFTP transfer time. My VMware VMs went from 1 minute 30 seconds to 30 seconds. My Hyper-V VMs went from 45 seconds to 15 seconds! My RealTek NICs in a HP box went from 6-8 minutes to 1 minute and 30 seconds (much better and an acceptable time, not as nice as other NICs but definitely useable now). This looks like a major oversight by the ConfigMgr development team to not have this configurable or at the very least set to a variable window size by default (especially since not an issue like this with WDS). At least this dll hack gives a valid workaround until Microsoft fixes it. Thank you very much for your insight! I don’t think I would have been able to find full fix on my own. Btw, how did you find which Hex value in the DLL to update using Hex Workshop? Most of the dll is not readable.
Guys, this is a fantastic conversation. I am having almost the exact same problem Brent originally had, with the added oddity that when the VMXNET3 VMs fail, I can almost always get it to work with one or two soft reboots.
Brent — I have made no changes to anything to correct this. Can you summarize the net changes you made to get it to work? I see the DLL editing, but would we also need to adjust certain registry keys? I assume this is all being done to the SCCM DP?
This would make a great instructional blog post.
Thanks very much — I’m hopeful this will correct our problem!
Chris, many changes were made during testing to narrow stuff done, but to actually reproduce the DLL fix scenario from a default environment is only a couple of steps. Make the following changes on the SCCM DP:
1) Modify DLL as mentioned in Patrick’s reply (modify HEX sequence of BA07000035 to BA08000035)
2) Create the following DWORD registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP\RamdiskTFTPBlockSize and set the value to 4 or 8 (8 is faster).
Note: Setting the labeled BlockSize key in registry is intentional, as this is key the smspxe.dll is looking for, the DLL hack changes how that key is interpreted.
One other note: When editing the DLL, make sure Windows Deployment Services is stopped, or saving of the DLL will fail with no error.
That’s it, you should be able to test using just those two changes steps. Let me know if any further questions. I will post back what I hear from Microsoft as I hear.
🙂 I actually just meant could you explicitly say what the values you are using are, but your post was much better for the misunderstanding lol!
Thanks very much! Changes made — my problem is somewhat intermittent so it’ll take some testing to know for sure, but fingers crossed!
It was fun watching this play out today, here and in the TechNet post… especially since I’ve been fighting this for a couple weeks now. You should add the DLL details to your Connect posting too… might get the attention of the product team guys easier. Could just be an oversight in the code.
Again — great job Brent and Patrick!
Okay I’m tired and that last post was technically meant for the TechNet thread, since that’s where the misunderstanding was. 🙂
Setting the WindowSize to 8 made my VMXNET3 VMs do the TFTP download reeeeeaaaalllly slowly, but setting it to 4 made them go really fast and, at least for now, consistently without error!
OK A few words about windowsize values.
when we set windowsize=n the TFTP server will send n consecutive blocks before stopping for an ACK. bootmgr.exe is able to handle the load but remember when bootmgr.exe runs it still relays on the UNDI pxe driver which might have some problems dealing with the burst of TFTP blocks. This situation can be seen specially in pure virtualized environments when the packets never touch a real wire (very fast transfers) and/or the TFTP server runs in fast hardware and it does not control the minimum inter-packet dely. Then a higher windowsize value in these situations could lead to errors and that to slower transfers.
There’s also the false idea that windowsize can only be 4 or 8, but windowsize can be any value <= 64 (limited by the client).
Care must be taken when testing high values if the server does not limit the minimum inter-packet delay; this might lead to net resource starvation.
Fortunately there's always a sweet spot for the windowsize value you just have to test and find it.
Best,
Patrick
Thanks. I also updated the connect post with further information. Hopefully someone at Microsoft will respond. In addition, I have a support case on the matter as well.
sorry; what’s the “connect” post?
I created a connect post here: https://connect.microsoft.com/ConfigurationManagervnext/feedback/details/1051565/cant-incrase-ramdisktftpwindowsize
I’m glad it did work. Please consider this is just for testing purposes absolutely not for production. The idea was just to prove that the SCCM client TFTP capabilities “are” really there; it is eventually MS who should add the TFTP windowsize option to smspxe.dll.
Best,
Patrick
Yup, this does tell us that there is no reason for the functionality to not be there. Microsoft just needs to add some very simple code to address it. I have reached out to Microsoft about this and hopefully can get them to fix/add this, just not sure how long that could take…Thanks.
I can confirm it works for me as well. Amazing work, thanks!
Great, glad to hear that others are seeing the same results as well.
When using Serva as a test, should I be changing the “Enforce Windowed TFTP, window-size” or “Parse ‘windowsize’ option limited to” ?
In Serva the windowsize parameter is controlled by
“Parse ‘windowsize’ limited to”
possible values e.g. from 4 to 32.
Hi Brent
Have you tried to make changes to the TFTP tab of the WDS properties?
There is an option to enable or disable Variable Window Extension.
Cheers
Felipe
if the code that should parse RamdiskTFTPWindowSize and inject its value in the dynamically created BCD is missing from smspxe.dll then no matter how many settings you might try; the created BCD will not include the windowsize parameter.
I have “grepped” the whole content of mu_system_center_2012_configuration_manager_and_endpoint_protection_with_sp1_x86_x64_dvd_1565907.iso and there’s not any component containing the string “RamdiskTFTPWindowSize”.
Best,
Patrick
Thanks Patrick.
Yes, I can confirm what Patrick already stated, changing that setting in WDS has no affect because BCD is handled by SCCM and smspxe.dll unfortunately does not parse the setting.
FYI to all, I got a reply from Microsoft on this issue. They officially have this classified as a “bug” with a DCR assigned and slated to be fixed in SCCM 2015. Unfortunately it looks like there will not be a fix in a SCCM 2012 CU. For now, the DLL hack is the only workaround until they update their code in the next version.
Still, good job to both of you for finding it!
Brent – would you mind emailing me your MS case number for that if you have one? I’d like to use it as a reference when I try to get them to issue us a private hotfix. There’s a chance for that given our size and the impact. Thanks!
Chris, the MS case number was: 114121112156755. Let me know if you make any progress in getting a private fix.
I’m also interested on this; if you get the fix please let me know (see my e-address at Serva’s site).
Best,
Patrick
I am also interested in finding a solution by Microsoft to this.
I implemented the DLL-changes according to Patrick and it speeded up the boot.wim transfer from 5 minutes to 1 minute 20 seconds. Still it could be faster, but this already is a great enhancement to the situation before.
In my organisation the optimal WindowSize was 8. WindowSize 4 was 1min 40sec, 16 was 1min 30sec.
BR,
Juha-Matti
Fore more details about windowsize behavior and its values you can take a look at RFC 7440
https://tools.ietf.org/rfc/rfc7440.txt
Best,
Patrick
I’ve just made this change on 2012 R2 CU4.. setting the value to 8 did get poor performance, but 4 is quick 🙂
FYI, I just tested this with ConfigMgr 2012 R2 SP1 (2012 SP2) and it appears that Microsoft has not yet fixed the issue by updating the DLL to be able to natively set the WindowSize via the registry. Also, it does appear that Microsoft has modified the DLL in 2012 R2 SP1 (2012 SP2), as the DLL I modified for 2012 R2 no longer works. I had to edit the dll manually again and create a new custom DLL based off the new original to make this fix work in 2012 R2 SP1 (2012 SP2). Searching for the same text in the hex editor as I did for the R2 dll did work.
Nice 🙂
Hi
Thanks I have come across your post and have not made the above changes as of yet. The issue I am having is only on one site that has a local DP however when PXE booting I am getting the Contacting Server : IP_Address . . . . . . . . .
This is intermittent after a few restarts it will eventually give you F12 option.
Looking in the PXE log all i get is below when I get this issue.
In SSL, but with no client cert SMSPXE 28/07/2015 22:53:47 4316 (0x10DC)
Client lookup reply:
SMSPXE 28/07/2015 22:53:50 4316 (0x10DC)
00:50:56:A9:AD:27, EBA32942-8C41-5075-E308-D7F8B247847B: device is not in the database. SMSPXE 28/07/2015 22:53:50 4316 (0x10DC)
Looking for bootImage CE2002D5 SMSPXE 28/07/2015 22:53:50 4316 (0x10DC)
In SSL, but with no client cert SMSPXE 28/07/2015 22:53:50 4316 (0x10DC)
In SSL, but with no client cert SMSPXE 28/07/2015 22:53:52 4316 (0x10DC)
Looking for bootImage CE2002D5 SMSPXE 28/07/2015 22:54:05 4316 (0x10DC)
In SSL, but with no client cert SMSPXE 28/07/2015 22:54:05 4316 (0x10DC)
In SSL, but with no client cert SMSPXE 28/07/2015 22:54:07 4316 (0x10DC)
Looking for bootImage CE2002D5 SMSPXE 28/07/2015 22:54:24 4316 (0x10DC)
In SSL, but with no client cert SMSPXE 28/07/2015 22:54:24 4316 (0x10DC)
In SSL, but with no client cert SMSPXE 28/07/2015 22:54:27 4316 (0x10DC)
Your help will be greatly appreciated
FYI, if anyone is using the custom DLL, I found one caveat with upgrading to SCCM 2012 R2 SP1 CU1. If you have the custom DLL in place on a remote distribution point, the Site Server will fail to update the DP on any remote server that is using the custom DLL, noting that it can’t validate the smspxe DLL. Before install CU1 it is important to remove the custom DLL, replace with the original one, upgrade to CU1, then put the custom DLL back in place.
sorry for bumping a post this old, but i cant find any answers.
We are running configmgr 2007 R3. After migrating confmgr to co-location and wmvare hosts the tftp transfer takes forever.
I´m trying to set RamDiskTFTPBlockSize to 1432 (thats what supported in vmware) in the registry and then I am changing smspxe.dll (X:\sms_ccm\x64\smspxe.dll) from BA07000035 to BA08000035 but it doesnt go any faster.
And if I check one of the autogenerated bcd files in smstemp it says this:
Setup Ramdisk Options
———————
identifier {ramdiskoptions}
ramdisksdidevice boot
ramdisksdipath \SMSBoot\boot.sdi
ramdisktftpwindowsize 1432
Any help whit this?
1) you should set RamDiskTFTPBlockSize to 8 (or 4) in the registry.
2) The created BCD should then include ramdisktftpwindowsize = 8 (or 4)
3) Now you boot a client.
Clients will limit ramdisktftpwindowsize to 64 if the BDC offers a bigger number. But a big number like 64 can many times lead to slow transfers. Please read the thread again.
Best,
Patrick
Yeah, i read it again after i posted and changed to 8, the result is slightly better, but it still takes around 15min. Think we might have package drops, will investigate further later when i have time
1) TFTP needs a good network.
2) you can also trim the value; probably 4 will lead to better results than 8.
I’m using vmware here and I have set RamDiskTFTPBlockSize to 8
To confirm, you are using the VMXNet3 card in the vm settings?
Just to also add that, its set to that as the after the hex edit the value is actually used as the TCPWindowSize and as the param isnt specified for the block size it defaults to an internal value around 1432 iirc.
This mod should not be affected by the used NIC driver at all.
It is the client’s bootmgr.exe/bootmgfw.efi the one that will interpret the BCD parameter ramdisktftpwindowsize and act accordingly when TFTP transferring i.e. Boot.wim
You are correct about adding RamDiskTFTPBlockSize to the registry in order to get its value added to the BCD as “TCPWindowSize” while the missing RamDiskTFTPBlockSize is replaced at the client by a default value.
Best,
Patrick
The only reason I state VMXNet3 is that I only reliably had the issue after changing my vm’s from the E1000 to the VMXNet3 driver – so something was worse when using the VMXNet3 🙂
I see, well VMware has some issues with TFTP.
i.e. you can see a report here
https://communities.vmware.com/thread/519250?sr=inbox
about how their new Worstation 12 client mistakenly set the TFTP block size “only” when using particular drivers. This thing started happening with Workstation 12.
You might be running into some related issue;
Following Bret info:
1) Modify DLL as mentioned in Patrick’s reply (modify HEX sequence of BA07000035 to BA08000035)
2) Create the following DWORD registry key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP\RamdiskTFTPBlockSize and set the value to 4 or 8 (8 is faster).
makes my Vmware VM not boot (it just hangs on .sdi load)
The
I can see the bcd used is (due to edit of the default.bcd):
Device options
————–
identifier {68d9e51c-a129-4ee1-9725-2ab00a957daf}
ramdisksdidevice boot
ramdisksdipath \Boot\Boot.SDI
ramdisktftpblocksize 1456
ramdisktftpwindowsize 8
ramdisktftpvarwindow Yes
And the above works, but slow
Awesome thread!
Just patched smspxe.dll from BA07000035 to BA08000035 and set RamDiskTFTPBlockSize=1432 globally.
Did some tests (BlockSize is set to 1432 in both cases)
Original dll:
SCCM boot.wim (250 Mb) is downloaded over 100Mpbs in 55 sec.
Modified dll:
Same file on the black screen is downloaded in 25 sec.
Replaced it everywhere via package (.cmd):
net stop WDSServer
IF EXIST C:\SMS_DP$ copy smspxe.dll C:\SMS_DP$\sms\bin /y
IF EXIST D:\SMS_DP$ copy smspxe.dll D:\SMS_DP$\sms\bin /y
IF EXIST E:\SMS_DP$ copy smspxe.dll E:\SMS_DP$\sms\bin /y
IF EXIST “C:\Program Files\Microsoft Configuration Manager\bin\X64” copy smspxe.dll “C:\Program Files\Microsoft Configuration Manager\bin\X64” /y
reg add HKLM\SOFTWARE\Microsoft\SMS\DP /v RamDiskTFTPBlockSize /t REG_DWORD /d 1432 /f
net start WDSServer
@i3laze
you are making a mistake:
After the patch the RamdiskTFTPBlockSize variable will in fact be interpreted by mspxe.dll as “RamdiskTFTPWindowSize” then RamdiskTFTPBlockSize should be set to any value between 4 and 8 or between 4 and 16.
If after the patch you do set RamdiskTFTPBlockSize = 1432 the booting client will limit RamdiskTFTPWindowSize to 64 but that high value many times leads to suboptimal transfer figures.
Best,
Patrick
I was misleaded by tons of “RamDiskTFTPBlockSize=1432” in the comments above.
So you suggest to
A. Remove the key from registry?
or
B. Patch it to 8?
just set RamDiskTFTPBlockSize=8 or 4 or 16 and see the results.
When you set RamDiskTFTPBlockSize=1432 the client was limiting the value to 64, if you saw transfer improvements in this situation then your network is good and windowsize=64 is not bad for you but this is just “your” case. For a more conservative approach just use 4/8/ or 16
Best,
Patrick
Got ~25sec at 8 and ~20sec at 16.
I’ll stay at 16, cause performance is fine.
reg add HKLM\SOFTWARE\Microsoft\SMS\DP /v RamDiskTFTPBlockSize /t REG_DWORD /d 16 /f
RamdiskTFTPBlockSize = 16 with hacked dll and corresponding setting in bcd, makes 450 Mb .wim boot in Optiplex 3020 in 10 seconds.
Same wim booting Hyper-V 2012 R2 VM with legacy netowork (only one that can do PXE boot) takes > 1 HOUR !!!
sebus
Yes; with the hack things gets much better
Be careful with VMware 12; this bug is now affecting TFTP transfers
https://communities.vmware.com/thread/519250
Best,
Patrick
Was updating from 2012R2 SP1CU1 to CU2.
Got upgrade error on all DPs : )
distmgr.log says:
“Error in verifying the trust of file ‘\\DistrPoint.FQDN.COM\SMS_DP$\sms\bin\smspxe.dll’.
Failed to install DP files on the remote DP. Error code = 16389”
Have to roll back original CU1 smspxe.dll for a while.
Just FYI.
Ahh yeah, I ran into that issue too, I meant to post an update here but I forgot. You have to roll DLL back before doing any SCCM upgrades, otherwise it will fail. Once the upgrades are complete you can replace it again. If smspxe.dll got changed by the update, then you need to edit it again, otherwise you can use the same edited DLL.
Got a collection with SCCM Distribution Points for this case : )
select * from SMS_R_System where SMS_R_System.ResourceNames in (Select ServerName FROM SMS_DistributionPointInfo)
I’ve created app deployment for my script above.
Detection method is “modify date le 11/01/2015 for smspxe.dll in 4 places C:\SMS_DP$ or D or E or C:\ProgramFiles paths”.
And another app deployment with script reverting original file back.
Not sure what is going on but after I upgraded to SCCM 2012 R2 SP1 and modified the DLL on my remote site it hangs on the SDI file when I am trying to image my Dell boxes. I have to put the original back which is taking about 15 minutes to complete. Has any one else had this problem
Sorry both my Block and Windows are set to 8
try with different clients; On rare occasions some clients present problems with windowed TFTP. Also check your traffic with wireshark and try to see how the sdi TFTP transfer fails.
Best,
Patrick
FYI, Microsoft has finally fixed this. Its available in Technical Preview 1603 of Configuration Manager and should also be in the next Current Branch release as well (after 1602). More info: https://technet.microsoft.com/en-us/library/mt595861.aspx#BKMK_RC1603
Is it confirmed? It looks like they finally did it; good job MS.
I can confirm this is out of preview and in version 1606! I have tested it and works great! Yay! 🙂
How do we use this in 1606? Is there a place in the console to enable it or tweak the settings?
Thanks.
Unfortunately there are no settings viewable in the UI. This is fully configurable via the registry. Simply open the Registry Editor, navigate to the following key: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\SMS\DP. Once there create a new DWORD for which setting you want to manage. For Window Size: RamDiskTFTPWindowSize or Block Size: RamDiskTFTPBlockSize. Then enter the size you want for the Window/Block sizes for the value field. I have found leaving RamDiskTFTPBlockSize default and changing RamDiskTFTPWindowSize to either 6 or 8 works best in my environment.