Friday, May 2, 2014

Be Like the NSA... on your Exchange server...

Long story short, we have a consultant who wants to create a report to figure out:
1) how often our customers (students) contact our dept
2) how long it takes for our employees to contact them back
3) who is responding to whom
4) some general e-mail numbers about productivity.

We tasked a student worker John ( Johnt5818@gmail.com )with figuring out if this was possible using Powershell and an Exchange module (with an active directory plugin too).  I helped him stand up the trial VHD of Exchange for him to play with, and he certainly delivered.

He found another person's script that didn't seem to work right, then fixed it and customized it for what we needed.  We needed all of the employees in our department to show up in the CSV, and we needed it placed on a remote server.  We do not manage the production Exchange environment here, so we tested it to run as a scheduled task to run every day or so.

script:
------------------------

<#
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
| Information added by John Thompson 4/30/2014                                                                                                             |
| Command to add exchange commands: add-pssnapin Microsoft.Exchange.Management.PowerShell.E2010                                                            |
| Script is a modification from here: http://gallery.technet.microsoft.com/scriptcenter/bb94b422-eb9e-4c53-a454-f7da6ddfb5d6/view/Discussions#content      |
|                                                                                                                                                          |
| When you run this script it will gather statistics on whatever switches below you have set to true. You need to specifiy what active Directory you want  |
| the script to run on as well. It will save all the infromation to the save Location that you specify.                                                    |
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
#>
<#
.NOTES (original)
Updated by Justin Beeden 11.25.2013
V1.0 
    I have updated the original script by parameterizing it and adding support for Exchange 2013, including the option to include 2013 Health mailboxes. 
    Original script written by Rob Campbell
        http://blogs.technet.com/b/heyscriptingguy/archive/2011/03/02/use-powershell-to-track-email-messages-in-exchange-server.aspx
.SYNOPSIS
Checks Message Tracking Logs and pulls message statistics for an Exchange Organization.  Works on Exchange 2007 and later.
By default does not include messages for the Exchange 2013 Health Mailboxes, use the Include2013HealthMailboxes to include them in report.
.DESCRIPTION
Checks Message Tracking Logs and pulls message statistics for an Exchange Organization.  Works on Exchange 2007 and later.
By default does not include messages for the Exchange 2013 Health Mailboxes, use the Include2013HealthMailboxes to include them in report.
.PARAMETER DaysToGoBack
Specifies the amount of days to go back and search through the Message Tracking Logs.
.PARAMETER IncludeDistListStats
Optional parameter that will add an additional exported csv containing statistcics for Distribution List usage.
.PARAMETER Include2013HealthMailboxes
Optional parameter that will include the Exchange 2013 Health Mailboxes used by Managed Availabilty in the exported csv report.
.EXAMPLE
PS> .\GetMessageStatsToFile.ps1 -DaysToGoBack 30
Searches the past 30 days of Message Tracking Logs. Message Trackign logs are kept for 30 days by default on all transport servers.
.EXAMPLE
PS> .\GetMessageStatsToFile.ps1 -DaysToGoBack 15 -IncludeDistListStats
Searches the past 15 days of Message Tracking Logs. Also exports optional DistListStats csv.
#>

#                                        Variables to change
#--------------------------------------------------------------------------------------------------------------------------------------------
[int]$DaysToGoBack = 365 # used to set how many days you want the script to look back on.
[switch]$IncludeDistListStats = $false # Not tested                                                        -Logic: line 304
[switch]$Include2013HealthMailboxes = $false # not tested                                                  -Logic: within $numberOfEmailsSent
[switch]$numberOfEmailsSent = $true # tracks the amount of emails sent and how much data with it.          -Logic: line 156
[switch]$ToFromEmailStats = $true # Set to [true] to get statistics on who received/sent messages to whom. -Logic: line 337

$activeDirectory = "DLS_DL" #Set to the active directory group that you want to have the stats for.                 -Logic: line 139

$SaveLocation = "\\SLC-DC01\Email_log" #the network folder you want to have everything saved to.

#--------------------------------------------------------------------------------------------------------------------------------------------
#checks if you have already added the PSsnapin. if you have not it will add it for you.
if ((get-pssnapin -name Microsoft.exchange.management.powershell.e2010 -ErrorAction SilentlyContinue ) -eq $null)
{
    add-pssnapin Microsoft.Exchange.Management.PowerShell.E2010  
}


IF ($true) #General variables and set up script
{
    $rundate = $(Get-Date).toshortdatestring()
    $startdate = $((Get-Date).adddays(-$DaysToGoBack)).toshortdatestring()
    $outfile_date = (Get-Date -Format "mm_dd_yyy-hh_mm").ToString() 

    $outfile = $SaveLocation + "\DaysMessageStats_" + $outfile_date + ".csv"
    $dloutfile = "DistListStats_" + $outfile_date +".csv"
    $ToFromOutFile = $SaveLocation + "\ToFromEmailStats_" + $outfile_date + ".csv"
 
    $accepted_domains = Get-AcceptedDomain |Foreach {$_.domainname.domain} 
    [regex]$dom_rgx = "`(?i)(?:" + (($accepted_domains |% {"@" + [regex]::escape($_)}) -join "|") + ")$" 
 
    $mbx_servers = Get-ExchangeServer |Where {$_.serverrole -match "Mailbox"}|Foreach {$_.fqdn} 
    [regex]$mbx_rgx = "`(?i)(?:" + (($mbx_servers |Foreach {"@" + [regex]::escape($_)}) -join "|") + ")\>$" 
 
    $msgid_rgx = "^\<.+@.+\..+\>$" 
 
    $hts = Get-TransportServer -WarningAction SilentlyContinue | Foreach {$_.name} 
 
    $exch_addrs = @{} 
    $msgrec = @{} 
    $bytesrec = @{} 
    $msgrec_exch = @{} 
    $bytesrec_exch = @{} 
    $msgrec_smtpext = @{} 
    $bytesrec_smtpext = @{} 
    $total_msgsent = @{} 
    $total_bytessent = @{} 
    $unique_msgsent = @{} 
    $unique_bytessent = @{} 
    $total_msgsent_exch = @{} 
    $total_bytessent_exch = @{} 
    $unique_msgsent_exch = @{} 
    $unique_bytessent_exch = @{} 
    $total_msgsent_smtpext = @{} 
    $total_bytessent_smtpext = @{} 
    $unique_msgsent_smtpext=@{} 
    $unique_bytessent_smtpext = @{} 
    $dl = @{} 
 
    $obj_table = {@" 
     Date = $rundate 
     User = $($address.split("@")[0]) 
        Domain = $($address.split("@")[1]) 
     Sent Total = $(0 + $total_msgsent[$address]) 
        Sent MB Total = $("{0:F2}" -f $($total_bytessent[$address]/1mb)) 
        Received Total = $(0 + $msgrec[$address]) 
        Received MB Total = $("{0:F2}" -f $($bytesrec[$address]/1mb)) 
        Sent Internal = $(0 + $total_msgsent_exch[$address]) 
        Sent Internal MB = $("{0:F2}" -f $($total_bytessent_exch[$address]/1mb)) 
        Sent External = $(0 + $total_msgsent_smtpext[$address]) 
        Sent External MB = $("{0:F2}" -f $($total_bytessent_smtpext[$address]/1mb)) 
        Received Internal = $(0 + $msgrec_exch[$address]) 
     Received Internal MB = $("{0:F2}" -f $($bytesrec_exch[$address]/1mb)) 
     Received External = $(0 + $msgrec_smtpext[$address]) 
     Received External MB = $("{0:F2}" -f $($bytesrec_smtpext[$address]/1mb)) 
     Sent Unique Total = $(0 + $unique_msgsent[$address]) 
     Sent Unique MB Total = $("{0:F2}" -f $($unique_bytessent[$address]/1mb)) 
     Sent Internal Unique  = $(0 + $unique_msgsent_exch[$address])  
     Sent Internal Unique MB = $("{0:F2}" -f $($unique_bytessent_exch[$address]/1mb)) 
     Sent External Unique = $(0 + $unique_msgsent_smtpext[$address]) 
     Sent External Unique MB = $("{0:F2}" -f $($unique_bytessent_smtpext[$address]/1mb)) 
"@ 
    } 
 
    $props = $obj_table.ToString().Split("`n")|% {if ($_ -match "(.+)="){$matches[1].trim()}} 
 
    $stat_recs = @() 
 
    function time_pipeline 
    { 
        param ($increment  = 1000) 
        begin{$i=0;$timer = [diagnostics.stopwatch]::startnew()} 
        process 
     { 
            $i++ 
            if (!($i % $increment))
      {
       Write-host “`rProcessed $i in $($timer.elapsed.totalseconds) seconds” -nonewline
      } 
            $_ 
        } 
        end 
     { 
            write-host “`rProcessed $i log records in $($timer.elapsed.totalseconds) seconds” 
            Write-Host "   Average rate: $([int]($i/$timer.elapsed.totalseconds)) log recs/sec." 
        } 
    } 
}

#------Gets all the email accounts for the Active Directory that you chose to run the script against.-----
    $emailArray = @()
    $counter = 0
    $accounts = Get-ADGroupMember $activeDirectory
    foreach ($account in $accounts)
    {
        $principalName = get-ADUser $($account.SamAccountName) -Properties mail
        $counter ++
        $emailArray += $principalName.mail#$accountObject
    }
    Write-Host "Running script for $counter accounts" -ForegroundColor Green
    Write-Host $emailArray -ForegroundColor Green




#-----------------------Mane body of logic--------------------------
    If ($numberOfEmailsSent)#
    { 
        foreach ($ht in $hts)
        { 
            Write-Host "`nStarted processing $ht" 
 
            get-messagetrackinglog -Server $ht -Start "$startdate" -End "$rundate 11:59:59 PM" -resultsize unlimited | time_pipeline |%{ 
          If ($Include2013HealthMailboxes)
          { 
           if ($_.eventid -eq "DELIVER" -and $_.source -eq "STOREDRIVER")
           { 
            if ($_.messageid -match $mbx_rgx -and $_.sender -match $dom_rgx) 
            {
             $total_msgsent[$_.sender] += $_.recipientcount 
             $total_bytessent[$_.sender] += ($_.recipientcount * $_.totalbytes) 
             $total_msgsent_exch[$_.sender] += $_.recipientcount 
             $total_bytessent_exch[$_.sender] += ($_.totalbytes * $_.recipientcount) 
             foreach ($rcpt in $_.recipients)
             { 
              $exch_addrs[$rcpt] ++ 
              $msgrec[$rcpt] ++ 
              $bytesrec[$rcpt] += $_.totalbytes 
              $msgrec_exch[$rcpt] ++ 
              $bytesrec_exch[$rcpt] += $_.totalbytes 
             }
            }  
            else 
            { 
             if ($_.messageid -match $msgid_rgx)
             {    
              foreach ($rcpt in $_.recipients)
              { 
               $msgrec[$rcpt] ++ 
               $bytesrec[$rcpt] += $_.totalbytes 
               $msgrec_smtpext[$rcpt] ++ 
               $bytesrec_smtpext[$rcpt] += $_.totalbytes 
              } 
             } 
            } 
           }
          }
          Else 
          {
           if ($_.eventid -eq "DELIVER" -and $_.source -eq "STOREDRIVER" -and $_.Recipients -notmatch "HealthMailbox")
           { 
            if ($_.messageid -match $mbx_rgx -and $_.sender -match $dom_rgx) 
            {
             $total_msgsent[$_.sender] += $_.recipientcount 
             $total_bytessent[$_.sender] += ($_.recipientcount * $_.totalbytes) 
             $total_msgsent_exch[$_.sender] += $_.recipientcount 
             $total_bytessent_exch[$_.sender] += ($_.totalbytes * $_.recipientcount) 
             foreach ($rcpt in $_.recipients)
             { 
              $exch_addrs[$rcpt] ++ 
              $msgrec[$rcpt] ++ 
              $bytesrec[$rcpt] += $_.totalbytes 
              $msgrec_exch[$rcpt] ++ 
              $bytesrec_exch[$rcpt] += $_.totalbytes 
             } 
            }  
            else 
            { 
             if ($_.messageid -match $msgid_rgx)
             { 
              foreach ($rcpt in $_.recipients)
              { 
               $msgrec[$rcpt] ++ 
               $bytesrec[$rcpt] += $_.totalbytes 
               $msgrec_smtpext[$rcpt] ++ 
               $bytesrec_smtpext[$rcpt] += $_.totalbytes 
              } 
             } 
            }
           }
          }
          If ($Include2013HealthMailboxes)
          {  
           if ($_.eventid -eq "RECEIVE" -and $_.source -eq "STOREDRIVER")
           { 
            $exch_addrs[$_.sender] ++ 
            $unique_msgsent[$_.sender] ++ 
            $unique_bytessent[$_.sender] += $_.totalbytes 
            if ($_.recipients -match $dom_rgx)
            { 
             $unique_msgsent_exch[$_.sender] ++ 
             $unique_bytessent_exch[$_.sender] += $_.totalbytes 
            }
            if ($_.recipients -notmatch $dom_rgx)
            { 
             $ext_count = ($_.recipients -notmatch $dom_rgx).count 
             $unique_msgsent_smtpext[$_.sender] ++ 
             $unique_bytessent_smtpext[$_.sender] += $_.totalbytes 
             $total_msgsent[$_.sender] += $ext_count 
             $total_bytessent[$_.sender] += ($ext_count * $_.totalbytes) 
             $total_msgsent_smtpext[$_.sender] += $ext_count 
             $total_bytessent_smtpext[$_.sender] += ($ext_count * $_.totalbytes) 
            }
           }
          }
          Else
          {
                    if ($_.eventid -eq "RECEIVE" -and $_.source -eq "STOREDRIVER" -and $_.Recipients -notmatch "HealthMailbox")
           { 
            $exch_addrs[$_.sender] ++ 
            $unique_msgsent[$_.sender] ++ 
            $unique_bytessent[$_.sender] += $_.totalbytes
            if ($_.recipients -match $dom_rgx)
            { 
             $unique_msgsent_exch[$_.sender] ++ 
             $unique_bytessent_exch[$_.sender] += $_.totalbytes 
            } 
            if ($_.recipients -notmatch $dom_rgx)
            { 
             $ext_count = ($_.recipients -notmatch $dom_rgx).count 
             $unique_msgsent_smtpext[$_.sender] ++ 
             $unique_bytessent_smtpext[$_.sender] += $_.totalbytes 
             $total_msgsent[$_.sender] += $ext_count 
             $total_bytessent[$_.sender] += ($ext_count * $_.totalbytes) 
             $total_msgsent_smtpext[$_.sender] += $ext_count 
             $total_bytessent_smtpext[$_.sender] += ($ext_count * $_.totalbytes) 
                        }                                
                    }        
                }
          if ($_.eventid -eq "expand")
          { 
           $dl[$_.relatedrecipientaddress] ++ 
                } 
            }      
     
        } 


        foreach ($address in $exch_addrs.keys)
        { 
            $stat_rec = (new-object psobject -property (ConvertFrom-StringData (&$obj_table))) 
            $stat_recs += $stat_rec | select $props 
        } 
 
        $stat_recs | export-csv $outfile -NoTypeInformation

        Write-Host "Email stats file is $outfile" -ForegroundColor Green  
    }
 
    If ($IncludeDistListStats) 
    { 
        If (Test-Path $dloutfile)
     { 
            $DL_stats = Import-Csv $dloutfile 
            $dl_list = $dl_stats | Foreach {$_.address} 
        } 
        else 
     { 
      $dl_list = @() 
      $DL_stats = @() 
        } 
     $DL_stats | Foreach{ 
                            if ($dl[$_.address])
                      { 
                                if ([datetime]$_.lastused -le [datetime]$rundate)
                       {  
                        $_.used = [int]$_.used + [int]$dl[$_.address] 
                        $_.lastused = $rundate 
                                } 
                            } 
        } 
     $dl.keys | Foreach  { 
                                if ($dl_list -notcontains $_)
                                { 
                                    $new_rec = "" | select Address,Used,Since,LastUsed 
                                    $new_rec.address = $_ 
                                    $new_rec.used = $dl[$_] 
                                    $new_rec.Since = $rundate 
                                    $new_rec.lastused = $rundate 
                                    $dl_stats += @($new_rec) 
                                } 
                            } 
        $dl_stats | Export-Csv $dloutfile -NoTypeInformation 
        Write-Host "DL usage stats file is $dloutfile" -ForegroundColor Green
    }

    #gets emails from who Emails where sent from and to whom. it will also display the subject line.
If ($ToFromEmailStats)
{
    $resultsarray = @()
    foreach ($server in $hts)
    {
        foreach ($emailaddress in $emailArray)
        {
            #grabs the all the information in the MessageTrackingLog with the desired parameters.
            #[if you need to limit what is being added into the result file filter it through this command]
            get-messagetrackinglog -Server $ht -Start "$startdate" -End "$rundate 11:59:59 PM" -resultsize unlimited | Where-Object {$_.sender -imatch $emailaddress} | time_pipeline |
          %{
                foreach ($email in $_)
                {
                   $resutlsObject = New-Object PSObject
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Sender" -value $($email.Sender | Sort-Object)
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Recipients" -value $($email.Recipients | Sort-Object)
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Event ID" -value $($email.EventId | Sort-Object)
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Time Stamp" -value $($email.Timestamp | Sort-Object)
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Client Host Name" -value $($email.ClientHostname | Sort-Object)
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Message Subject" -value $($email.MessageSubject | Sort-Object)
                   $resutlsObject | Add-Member -membertype NoteProperty -name "Total Bytes" -value $($email.totalbytes | Sort-Object)
    
                   $resultsarray += $resutlsObject
                }
            }
        }
        $resultsarray | Export-Csv $ToFromOutFile -NoTypeInformation
        Write-Host "Email stats file is $ToFromOutFile" -ForegroundColor Green
    }
}
-----------------------------------

Wednesday, April 2, 2014

Tripple boot mac, yet another web page on it.....

At work, they have given me a hand-me-down three year old mac with a broken screen to use as my laptop.  Making the best of things, I wanted to tripple boot it with OSX mavericks, Windows 8.1, and Kali Linux.


I won't get into great detail about the steps I took, as you can find a bazillion other sites to explain most of the process.  I could not find a solution to a few spots, so I will elaborate on those.


1)Installed mavericks, install REFIND  (the newer version of reFIT)
2)Use "bootcamp assistant"... BUT INTERRUPT BOOT TO WINDOWS, BOOT TO OSX INSTEAD.
3)Open disk utility, and delete the bootcamp partition, create two MORE partitions.  Format them in exFAT, call them BOOTCAMP and "whatever you want your Linux partition to be called for now."
 4) reboot, booting into windows using a DVD  (I know... but I couldn't get the USB to boot properly)
5) in windows, select the bootcamp partition.  it will say "cannot install here" or some such thing.  FORMAT that partition, then windows will be ok with it and will install.
6) update windows, if you want, or don't.... whatever.
7) after a reboot, make sure OSX still works.  (if REFIND doesnt pop up, you can hold down "option" at the "mac tone" and force it to boot to something other than windows.  Reinstall REFIND to make it work again.)
8) boot the machine again to kali.
9)If you miss the grub menu, it might fail... it did on me.
10) so reboot it again, this time, select "text install"
11) at the point where you want to select the HDD or storage, make sure to do it "manually"
12) find the partition you created for LINUX, and remove it. 
13) then create a partition that is "do not use" at the "beginning of the free space" of 1MB
14) create two more partitions, one that is EXT3 or whatever you like, mount it as "/".  the second one needs to be for SWAP.
15) Finish installing it.
16)  if it boots, you are good, skip to the later point if you windows drive will not boot.
17) when it doesn't boot (mine said "waiting for /dev to populate" forever), catch the grub menu, and do the failsafe option.  If it does still not boot, add "nouveau.modeset=0" at the end of the line that says "ro quiet" .
18) connect it to the Internets, then perform an apt-get update, and upgrade, and dist-upgrade.  This may solve part of your issue.  if not.....
19) install the nvidia drivers:
"apt-get install -y linux-headers-$(uname -r)
apt-get install nvidia-kernel-dkms"
My install took care of uninstalling the "generic" drivers, as listed on other web help pages.
20)  Now that you have a kali install that works, let's boot into windows.
21) if you are staring at an error or a blinking cursor like I was, the boot partion/UFEI is all messed up.
22) Boot to OSX
23) install gdisk (gpt-disk)
24) run sudo gdisk /dev/diskx....   I only had one drive, so it was sudo gdisk /dev/disk0
25) type "b" to back up the partition table
26) now hit "r" and "p".  take note of the partition numbers of the three installs.
27) Enter ‘h’ to recreate your hybrid MBR. Enter each partition number you want (mine were 2 4 6). It will then ask you for a bunch of settings:
Place an EFI partition first: yes/y; For each partition, make bootable: no; For each partition, hex values: AF for OS X, 07 for Windows, 83 for Linux.
28) type "w" to write it.
29) reboot once, just to be sure, go back to OSX
30) if REFIND didn't appear, reinstall that too.
31) enjoy your mac, even though it only runs OSX 30% of the time.

 -AR
"Your mileage may vary, I only wrote this to remind myself how I did it, so if you read it and do something stupid to your own box I take no responsibility.  This is a private note to myself that I have published publicly."

REFERENCES:
http://www.rodsbooks.com/refind/
http://docs.kali.org/general-use/install-nvidia-drivers-on-kali-linux
http://irvanjit.wordpress.com/2013/06/30/triple-booting-mac-os-x-10-8-mountain-lion-windows-8-and-ubuntu-13/