Mike Says Meh The blog of Sys Admin Mike Kauspedas

21May/150

Powershell to get all MAC addresses of each virtual machine on a host and export to tab delimited file

This will spit out the MAC address and name of each virtual machine on a host to a text file. You can then clean the file up and throw it in Excel. I had to use this to clean up duplicate MACs in two different clusters because I'm a bad sys admin and didn't change the MAC address pools on the hosts or SCVMM.

Get-VM|Get-VMNetworkAdapter | Select-Object VMName, MACAddress | Export-Csv -delimiter "`t" -path C:\macs.txt

It comes out looking like this and I just use a find and replace to remove all the " so I can throw it in Excel. Then I use Excel to find and sort duplicates.

#TYPE Selected.Microsoft.HyperV.PowerShell.VMNetworkAdapter
"VMName" "MacAddress"
"server01" "00155D0167E2"
"server02" "00155D0A65E1"

After you paste it into Excel you can use Conditional Formatting > Highlight Cell Rules > Duplicate Values. Then sort by the column with MAC addresses A>Z and look for the highlighted rows.

Filed under: Tech No Comments
21May/150

Use powershell to turn off IPSec Task Offloading on all Virtual Machines on a host

I know you came here to copy and paste so here it is. Run this on each host. Works on stand alone and in clusters.

Get-VM | Get-VMNetworkAdapter | Set-VMNetworkAdapter -IPsecOffloadMaximumSecurityAssociation 0

Very simple. Gets all virtual machines on a host, then their network adapters, then disables IPSec task offloading. This is equivalent to unchecking the box in the settings below.

You can also change that 0 (disable) to a different number to change the offload tasks. IE 333 or something. Not that I recommend that, I just like the number 3.

But wait, you only want to disable this on certain machines, and your machines all have similar names? Maybe you have virtual machines named MyVirtualMachine001 - MyVirtualMachine099. Here's the command with a select-object. Just change the name to whatever fits your environment and virtual machine names.

Get-VM | Where-Object {$_.Name -like "MyVirtualMachine0*"} | Get-VMNetworkAdapter | Set-VMNetworkAdapter -IPsecOffloadMaximumSecurityAssociation 0

Why disable this you ask? Well in my case we were having an issue with RDP being really slow on some virtual machines. We originally thought it was disk IO, but our IO numbers didn't add up. They just weren't bad enough to make a machine as slow as it was. Then we logged into console and realized that was a LOT faster. One of the engineers at work found this and we started testing by disabling. RDP was much faster after disabling IPSec task offloading.

Filed under: Tech No Comments
2May/150

The mother of all disk benches! 4 disk SSD array, server SSD array, storage spaces

I built a new disk array for my Hyper-V lab at home. Overkill? Yeah, but I got tired of waiting for VM's to boot, or build from deployment. When I study or learn something new I need efficiency! Plus there was a sale on 256GB drives and I found a little 4 disk hot swap box. For comparison I have a SSD array in a server, some single disks, and a 4 disk old SATA300 using Windows 8.1 storage spaces (software RAID).

My PC: Intel Core i5 4670K | 32GB DDR1600
I'm using Crystal disk mark for testing.

*Note: I use RAID 0, there is a write penalty but these are SSDs. Yes, data isn't important so redundancy, well, who cares. Although I do back it up with Backblaze which allows me to backup meaningless VHDs.
4 disk RAID 0 SSD Array. Sandisk SDSSDHP256G 256GB SATA600 SSD
Sequential Read (Q= 32,T= 1) : 1578.833 MB/s
Sequential Write (Q= 32,T= 1) : 1272.100 MB/s
Random Read 4KiB (Q= 32,T= 1) : 349.640 MB/s [ 85361.3 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 318.789 MB/s [ 77829.3 IOPS]
Sequential Read (T= 1) : 1419.964 MB/s
Sequential Write (T= 1) : 1146.151 MB/s
Random Read 4KiB (Q= 1,T= 1) : 125.328 MB/s [ 30597.7 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 100.152 MB/s [ 24451.2 IOPS]

Notice the increased read performance between the array and a single SSD disk, but how large the write penalty is with RAID 0.

Single SSD SATA600 OCZ-Agility 256GB
Sequential Read (Q= 32,T= 1) : 386.957 MB/s
Sequential Write (Q= 32,T= 1) : 209.348 MB/s
Random Read 4KiB (Q= 32,T= 1) : 262.283 MB/s [ 64033.9 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 211.379 MB/s [ 51606.2 IOPS]
Sequential Read (T= 1) : 187.703 MB/s
Sequential Write (T= 1) : 211.817 MB/s
Random Read 4KiB (Q= 1,T= 1) : 18.627 MB/s [ 4547.6 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 131.864 MB/s [ 32193.4 IOPS]

Single SATA Hybrid SATA600 ST750LX003-1AC154 750GB
Sequential Read (Q= 32,T= 1) : 104.197 MB/s
Sequential Write (Q= 32,T= 1) : 105.733 MB/s
Random Read 4KiB (Q= 32,T= 1) : 1.317 MB/s [ 321.5 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 1.199 MB/s [ 292.7 IOPS]
Sequential Read (T= 1) : 104.002 MB/s
Sequential Write (T= 1) : 106.107 MB/s
Random Read 4KiB (Q= 1,T= 1) : 0.663 MB/s [ 161.9 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 1.214 MB/s [ 296.4 IOPS]

Storage Spaces two-way mirror (RAID 10) 4 disk array SATA300 ST3750640AS 750GB
Sequential Read (Q= 32,T= 1) : 75.898 MB/s
Sequential Write (Q= 32,T= 1) : 24.826 MB/s
Random Read 4KiB (Q= 32,T= 1) : 0.649 MB/s [ 158.4 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 0.284 MB/s [ 69.3 IOPS]
Sequential Read (T= 1) : 75.491 MB/s
Sequential Write (T= 1) : 26.843 MB/s
Random Read 4KiB (Q= 1,T= 1) : 0.651 MB/s [ 158.9 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 0.284 MB/s [ 69.3 IOPS]

RAID 5 with 6 disk + hot spare. PERC H710P Mini SATA600
Sequential Read (Q= 32,T= 1) : 2855.447 MB/s
Sequential Write (Q= 32,T= 1) : 2016.346 MB/s
Random Read 4KiB (Q= 32,T= 1) : 349.066 MB/s [ 85221.2 IOPS]
Random Write 4KiB (Q= 32,T= 1) : 115.260 MB/s [ 28139.6 IOPS]
Sequential Read (T= 1) : 2185.178 MB/s
Sequential Write (T= 1) : 1794.224 MB/s
Random Read 4KiB (Q= 1,T= 1) : 60.030 MB/s [ 14655.8 IOPS]
Random Write 4KiB (Q= 1,T= 1) : 59.278 MB/s [ 14472.2 IOPS]

I'm going to have to work on the random reads and writes on this server. That should blow my home array out of the water.

Filed under: Tech No Comments
8Apr/150

Open your hosts file in notepad as administrator from command prompt

Working for a web hosting company I edit my hosts file a LOT. And forever I did it the hard way, right click on notepad and then open as administrator then open the hosts file. Unacceptable! I need my hosts file open and ready in a split second. Time is money, money is time, time is infinite, money is power, power corrupts, illuminati! But I digress. Here are the steps.

1. Open command prompt as administrator. Right click on it and open as administrator. Or go to your start screen and search for cmd then hold down CTRL + SHIFT + ENTER. I have classic shell installed because over the years I grew to love and admire the start menu and I refuse to give it up despite everyone's love of the new start tile metro screen.

Note: You'll know it's open as Administrator because the top of the window will say so and the default path will be C:\Windows\System32>. If you open command prompt as your normal user it will open to a default path of your user profile. IE C:\Users\user.name>

2. Type "notepad C:\Windows\System32\drivers\etc\hosts". Without the "" into the command prompt.

3. Now the hosts file is open and notepad is running as administrator.

. Yay!

Filed under: Tech No Comments
9Jan/151

How to remove the AMD PCI Express (3GIO) Filter Driver

tl;dr:

  1. Go to device manager.
  2. Find AMD PCI Express (3GIO) Filter Driver.
  3. Update the driver.
  4. Browse my computer.
  5. Point to the folder full of intel chipset drivers.
  6. Voila.
  7. Reboot.

My Mom's HP laptop just died, the night before she is scheduled to fly up to Canada for an extended visit. She needs the laptop, it's her livelihood. So I do what I think is the easiest approach and swap the drive into my laptop. Even knowing it's Windows 7 and I'm going from AMD to Intel based chipsets it should work. And it did. Except for PoS AMD PCI Express (3GIO) Filter Driver.

After swapping the drive I got all the drivers installed and showing good except my Intel HD Graphics 4000 driver. It had a big fat yellow ! and an error stating "This device cannot find enough free resources that it can use. (Code 12)". I made sure all the AMD drivers were uninstalled, cleaned the registry of their existence, and even searched for their old brethren ATI. After some quick searching through the rest of the drivers in device manager I found the last remaining AMD driver, AMD PCI Express (3GIO) Filter Driver. Right clicked, uninstall and check the box to uninstall the driver software. Turns out that is the PCI bus and when you uninstall the PCI bus it reinstalls everything else. So I patiently waited and rebooted thinking it would simply disappear and my HD 4000 would work.

Nope, because the AMD PCI Express (3GIO) Filter Driver is full of hate and demons and terrorizes laptops like a suicide bomber. Except it's one that keeps coming back and exploding and you never die but you feel the pain over and over and over again. Sorry, it's 2am right now ...

Anyway. After reading about other peoples' woes (all prior to 2012, and it's 2015 now) I discovered that AMD in their infinite wisdom had renamed the pcisys driver. This replaced the PCI bus driver. So what if I just tried to update the driver for the AMD PCI Express (3GIO) Filter Driver in device manager? Hrm? So I did but I pointed it to my folder full of intel chipset drivers and BAM, it reverted to PCI bus, rebooted, and voila.

Filed under: Tech 1 Comment
7Oct/140

Dell DSET report default password

The default password is "dell" without the "".

I recently ran a DSET report on an older server. Dell DSET is their debug utility that pulls hardware specs but more importantly hardware logs. So when your old PowerEdge has a blinking amber light you can run that report and figure out what's going on. When you run DSET it places a zip file on your desktop. This has a small HTML app that has the report, it looks a lot like OpenManage. In the report you will find the hardware logs and what is going on with the server. When you extract the file use the password dell.

dell dset

Filed under: Tech No Comments
6Oct/140

How to fix “Windows NT user or group servername\Administrators not found” in MSSQL

Trying to add the local administrators group to a SQL server with sys admin (sa) server roles? Getting the error below? I have a very simple fix.

Windows NT user or group 'COMPUTERNAME\Administrators' not found. Check the name again. (Microsoft SQL Server, Error: 15401)

sql admins sa

Instead of adding "COMPUTERNAME\Administrators" change it to "BUILTIN\Administrators" and it will work just find. Just erase your computer/server name and replace with BUILTIN.

sql builtin

This fix should work for SQL Server 2005, 2008, 2012, and 2014.

Filed under: Tech No Comments
1Oct/140

How to fix – WordPress Upgrade Download failed. SSL certificate problem: unable to get local issuer certificate

Getting this error when trying to upgrade WordPress?

Download failed.: SSL certificate problem: unable to get local issuer certificate

Do this.

1. Download the cert.pem file from here. Right click on the link and click save link as.

cert

2. Place the cert.pem file in your php folder.
IE: C:\Program Files (x86)\PHP\v5.5\cacert.pem

3. Edit the php.ini and search for "curl". Edit the curl.cainfo line with the following.
curl.cainfo = "C:\Program Files (x86)\PHP\v5.5\cacert.pem"
Save the php.ini file.

php ini

4. Now try your upgrade again. I didn't have any issues after performing those steps.

Filed under: Tech No Comments
29Apr/140

What events to search for to find a server reboot

With virtualization on the rise we sys admins find ourselves managing a lot more server than normal. Gone are the days of managing a couple racks of pizza boxes. Instead one of those pizza boxes may hold a hundred virtual servers itself. And with so many servers, and clients doing the same fun things, we find ourselves looking into the random "my server rebooted why?" question.

When investigating a reboot you can search the system event log for the event ID's below. Each one corresponds to a reboot and will help determine why. And after you find the actual reboot you can check the rest of the events around that time to see if anything lead to or caused it. For example windows updates, or a BSOD.

1074
The process Explorer.EXE has initiated the restart of computer SERVER01 on behalf of user SERVER01\UserName for the following reason: Other (Planned)
Reason Code: 0x85000000
Shutdown Type: restart
Comment: Server updates
6006
The Event log service was stopped.
6005
The Event log service was started.
109
The kernel power manager has initiated a shutdown transition.
19
Installation Successful: Windows successfully installed the following update: Definition Update for Windows Defender - KB2267602 (Definition 1.173.438.0)22
Restart Required: To complete the installation of the following updates, the computer will be restarted within 15 minutes:
13
The operating system is shutting down at system time.12
The operating system started at system time.
I just copy and paste this line into the event filter.
12,13,19,22,109,1074,6005,6006
Reboot Events
Filed under: Tech No Comments
25Apr/142

How to fix Event ID 2213 for DFSr

I work a lot with DFSr because we use it to keep some web farm replicated and some of our customer's private farms. I can tell you it sucks, it always breaks, and it's very hard to maintain. Although I'll caveat that by saying we probably shouldn't use it for web farms with millions of little files. Seems to work fine for AD. Anyway, this is the most common issue you will run into with DFSr, the unexpected crash or shut down. Both the nodes this occurred on did not crash, in fact they didn't even reboot or shut down. But that doesn't matter, DFSr still crashed. Below is just one example and the fix for it. It's obvious from the event what you need to do, but lets review anyway.

The one thing you HAVE to remember is to leave it alone. Do not touch it after you resume replication. That's the #1 mistake I see people making with troubleshooting DFSr. Either rebooting the server or restarting the server. DFSr keeps a journal (database) of all the changes to the replicated folders. You can't just restart the service or reboot the server to fix this. That's like trying to restart SQL to recover a corrupted database. Instead you need to recover that journal, which fortunately Microsoft tells you exactly how to do in the event log.

To get to the event log go to Control Panel --> Administrative Tools --> Event Viewer --> Applications and Services Logs --> DFS Replication.

Event ID 2213
The DFS Replication service stopped replication on volume C:. This occurs when a DFSR JET database is not shut down cleanly and Auto Recovery is disabled. To resolve this issue, back up the files in the affected replicated folders, and then use the ResumeReplication WMI method to resume replication.
Recovery Steps
1. Back up the files in all replicated folders on the volume. Failure to do so may result in data loss due to unexpected conflict resolution during the recovery of the replicated folders.
2. To resume the replication for this volume, use the WMI method ResumeReplication of the DfsrVolumeConfig class. For example, from an elevated command prompt, type the following command:
wmic /namespace:\\root\microsoftdfs path dfsrVolumeConfig where volumeGuid="32A74A78-0B49-11E2-93EE-806E6F6E6963" call ResumeReplication

You will need to run the command given in step two from the event in command prompt as administrator to resume replication. Remember that each node in the DFSr replication group has a different GUID. Get the command from event viewer on each node and run it. Example below.

wmic /namespace:\\root\microsoftdfs path dfsrVolumeConfig where volumeGuid="32A74A78-0B49-11E2-93EE-806E6F6E6963" call ResumeReplication

After you run it you will see Event ID 2212 in the log.

The DFS Replication service has detected an unexpected shutdown on volume C:. This can occur if the service terminated abnormally (due to a power loss, for example) or an error occurred on the volume. The service has automatically initiated a recovery process. The service will rebuild the database if it determines it cannot reliably recover. No user action is required.

You may also see Event ID 2218

The DFS Replication service is in the second step of replication database consistency checks after an unexpected shutdown. The database will be rebuilt if it cannot be recovered. No user action is required.

Now you just need to wait for the database to recover. Depending on the amount of files and how long it has been down for it can take a few minutes, several hours, or even days. You MUST leave it alone. Do not reboot the server or restart DFSr. That will simply start the process all over again.

Once it is fully recovered you will see event ID 2214.

The DFS Replication service successfully recovered from an unexpected shutdown on volume C:.This can occur if the service terminated abnormally (due to a power loss, for example) or an error occurred on the volume. No user action is required.

Once you see that event you are good to go. More info in this MS KB.

You may also want to see this list of hotfixes for DFSr for Windows 2008 and 2008 R2.

Filed under: Tech 2 Comments