PoshCode Logo PowerShell Code Repository

ScriptSVN.ps1 by Bergle 35 months ago
embed code: <script type="text/javascript" src="http://PoshCode.org/embed/927"></script>download | new post

Script from SVN Utility

This script processes template files and produces and output file.
Its a simple macro processor really.
We use it to extract the latest version of our database procedurrads and functions
from our source code control server to create deployment scripts.

This will assume you have svn command line client in your path.
This currently cannot authenticate with svn to retrieve data. So just simple svn: supported.
This is written and tested with Microsoft Windows Powershell v1.0.

  1. #
  2. # Script from SVN Utility
  3. # ~~~~~~~~~~~~~~~~~~~~~~~
  4. # This script processes template files and produces and output file.
  5. # Its a simple macro processor really.
  6. # We use it to extract the latest version of our database procedurrads and functions
  7. # from our source code control server to create deployment scripts.
  8. #
  9. # This will assume you have svn command line client in your path.
  10. # This currently cannot authenticate with svn to retrieve data. So just simple svn: supported.
  11. # This is written and tested with Microsoft Windows Powershell v1.0.
  12. #
  13. # When run it will look for all files with ".dro" extension in the current
  14. # directory and process them. If the ".dro" file does not specify the
  15. # OuputFile directive then the output file will have the same name as the
  16. # input file with ".dro" replaced by ".out"
  17. #
  18. # GETTING STARTED
  19. # ~~~~~~~~~~~~~~~
  20. # Go to a directory containing a .dro file and execute "ScriptSVN.ps1" It will process all .dro files it finds.
  21. # Look at $exampleInputFile below to see a sample input file content.
  22. #
  23. # This example will fail to get this example to work you will need
  24. #       1. SVN Server called sccserver.
  25. #   2. Files under the following path on SVN server
  26. #               svn://sccserver/trunk/database/maindb/Procedure/ProcedureA.sql
  27. #               svn://sccserver/trunk/database/maindb/Procedure/ProcedureB.sql
  28. #               svn://sccserver/trunk/database/maindb/Procedure/ProcedureD.sql
  29. #       3. The ProcedureD file must have a revision 43 available.
  30. #
  31. # File Format SYNTAX
  32. # ~~~~~~~~~~~~~~~~~~
  33. # 1.    '#' marks start of script comment
  34. #               Comments characters are converted to $comment on output to match output format.
  35. #               Currently understands Single Quote strings and will mostly not identify a # in a string as a comment.
  36. # 2.    '{' '}' mark start and end of a script directive.
  37. #               Only recognised if the '{' is first non white space on a line. [simplifies search match a lot]
  38. #               Only recognised if both start and end on same line.
  39. #               The result of the directive is output to the file.
  40. #               The output result is bracketed before and after by comments identifying the directive and parameters.
  41. #               White space preceding a directive is prepended to each line output from directive [maintains indenting].
  42. #               Extra data following a directive is completely ignored at the moment.
  43. # 3.    Other lines are output as they are
  44. #
  45. # A directive may have parameters.
  46. # Directive parameters must use double quotes to quote things with spaces.
  47. # For simplicity will always end up with a newline as last char in file.
  48. # Simple understanding of single quoted strings wont find comment characters inside single quoted strings.
  49. # Single quoted strings are supported because this was primarilly setup for SQL rollout scripts.
  50. # Double quoted strings are not supported anywhere but inside directives.
  51. #
  52. # DIRECTIVES
  53. # ~~~~~~~~~~
  54. # OutputFile
  55. #       set the name of file to write output.
  56. #       No lines that produce output in file must precede the line its on [so make it first]
  57. #       Or only after SourceControl line.
  58. # SourceControl
  59. #       Set the source control access string base path.
  60. #       Must have trailing slash [ no it does not check it has it ]
  61. #       eg.  svn://scc/project/trunk/Database/DBName/ for DBName procedures
  62. # Reference
  63. #       Extract a file from SourceControl with given path. eg. path  "Procedures/spnEmailMessageInsert.sql"
  64. #       First parameter is path to file to fetch from source control.
  65. #       Second parameter is an optional revision to retrieve of that file
  66. #       Third paremter is an optional string to output to file as well [not sure how useful this is]
  67. #               Inside this third parameter the string is processed for some special characters
  68. #               "\n" represents a new line. useful for "GO\n".
  69. #               "\t" represents a tab.
  70. #               The conversion only occurs for output data not for the trace data output to file.
  71. #
  72. # Example. Build.cmd file that converts the .dro file(s) into our file
  73. #       "powershell C:\Scripts\rollscript.ps1"
  74. #
  75. # HISTORY
  76. # 2008/12/11 rluiten    Created.
  77. # 2008/12/15 rluiten    Setup a v3.2 folder to test and fixed a bug or three.
  78. #                                               Useful now maybe.
  79. # 2008/12/17 rluiten    Added explicit revision retrieval to Reference.
  80. #                                               Reference now looks up the revision to be able to output to file.
  81. # 2009/02/23 rluiten    Corrected end of line outputs to CRLF.
  82. #
  83. # SOME EXTRA NOTES
  84. # ~~~~~~~~~~~~~~~~
  85. # We export our procedures and functions as a unit that drop themseles if they exist
  86. # create themselves and then set permissions. This is the functional block of code we manage
  87. # and track in our source code control server. This allows us to export the code from SQL again
  88. # with another small script and do a file comparison to check we havnt missed a change or
  89. # something hasnt snuck out to production without being in source control.
  90. #
  91. # Dont put identation in front of a {Reference} directive to a procedure as it will add indenting
  92. # to the start of every line and your exported code if you wish to diff your database back to
  93. # your source code control server will cause every line to be different.
  94. #
  95. # This utility is general and could be used for other stuff than SQL.
  96. # However the single quote processing of strings not inside directives may limit some scenarios.
  97. # This is the first script I have written so probably lots of things I can do better.
  98. # Happy to have feedback.
  99. # Hope someone finds this useful saves us here a lot of time keeping our deployment scripts
  100. # correct as we work.
  101. #
  102. $exampleInputFile = @"
  103. {SourceControl "svn://sccserver/trunk/database/maindb/"}
  104. {OutputFile "Test2.sql"}
  105. #{hello world}
  106.  
  107. {Reference "Procedure/ProcedureA.sql" }
  108. {Reference "Procedure/ProcedureB.sql" "" "print 'dont forget to do job 222'\n"}
  109. #{Reference "Procedure/ProcedureC.sql" } -- commented out for now
  110. {Reference "Procedure/ProcedureD.sql" "43"}     -- extract specific revision file
  111.  
  112. select 'This is a test'
  113. select 'What a nice day.'
  114.         #{Me to}
  115. # not here
  116. -- Zzzz ' this and # that '  # and here
  117. -- Aaaa# ' foiousf ds'  # and here
  118. -- Bbbb # ' foiousf ds  # and here
  119. --select 'Yyyy'
  120. "@
  121.  
  122. $commentStart = "#"
  123. $comment = "--"
  124. $version = "3"
  125.  
  126. # directives must be first non white space on line and not be empty.
  127. # doesnt matter what follows a directive
  128. # the content of the directive will always be group 3 of regex.
  129. # group 4 is data following directive
  130. $matchDirective=[Regex]"^(\s*)({([^{]+)})(.*)$"
  131.  
  132. # matching only single quote strings for now - and assume a string goes from first quote on line to last quote on line.
  133. # groups 0 whole match, 1 - before string, 2 - string, 3 - after string
  134. $matchSingleQuoteString=[Regex]@"
  135. ^([^']*)(['].*?['])([^']*)$
  136. "@
  137.  
  138. # returns array.
  139. # array 0 - prefix string to directive. [will always be whitespace]
  140. # array 1 - directive text inside brackets
  141. # array 2 - post fix data after directive
  142. function split-directive([string]$str)
  143. {
  144.         $myMatches = $matchDirective.match($str)
  145.         if ($myMatches.Success)
  146.         {
  147.                 return $myMatches.Groups[1], $myMatches.Groups[3], $myMatches.Groups[4]
  148.         }
  149. }
  150.  
  151. # return the index of the comment character on the line. -1 if no comment on line
  152. function get-commentindex([string]$str)
  153. {
  154.         $startCommentIndex = -1
  155.         $myMatches = $matchSingleQuoteString.match($str)
  156.  
  157.         # check for comment char before and after string in group 1 and 3.
  158.         if ($myMatches.Success) # found a string in our line so look around it for comments
  159.         {
  160.                 $commentIndex = ([string]$myMatches.Groups[1]).IndexOf($commentStart)
  161.                 if ($commentIndex -gt -1)
  162.                 {       # comment in group 1
  163.                         $startCommentIndex = $myMatches.Groups[1].Index + $commentIndex
  164.                 }
  165.                 else
  166.                 {
  167.                         $commentIndex = ([string]$myMatches.Groups[3]).IndexOf($commentStart)
  168.                         if ($commentIndex -gt -1)
  169.                         {
  170.                                 $startCommentIndex = $myMatches.Groups[3].Index + $commentIndex
  171.                         }
  172.                 }
  173.         }
  174.         else    # no string so just first index of comment
  175.         {
  176.                 $startCommentIndex = $str.IndexOf($commentStart)
  177.         }
  178.         return $startCommentIndex
  179. }
  180.  
  181. # returns array
  182. # array 0 - non comment part of line
  183. # array 1 - comment part of line including comment char
  184. function split-comment([string]$str)
  185. {
  186.         $startCommentIndex = get-commentindex $str
  187.         if ($startCommentIndex -gt -1)
  188.         {
  189.                 return $str.Substring(0, $startCommentIndex), $str.Substring($startCommentIndex)
  190.         }
  191.         else
  192.         {
  193.                 return $str
  194.         }
  195. }
  196.  
  197. ## got from http://poshcode.org/496 thank you Jaykul :)
  198. ################################################################################
  199. ## Convert-Delimiter - A function to convert between different delimiters.
  200. ## E.g.: commas to tabs, tabs to spaces, spaces to commas, etc.
  201. ################################################################################
  202.         ## Written primarily as a way of enabling the use of Import-CSV when
  203.         ## the source file was a columnar text file with data like services.txt:
  204.         ##         ip              service         port
  205.         ##         13.13.13.1      http            8000
  206.         ##         13.13.13.2      https           8001
  207.         ##         13.13.13.1      irc             6665-6669
  208.         ##
  209.         ## Sample Use:  
  210.         ##    Get-Content services.txt | Convert-Delimiter " +" "," | Set-Content services.csv
  211.         ##         would convert the file above into something that could passed to:
  212.         ##         Import-Csv services.csv
  213.         ##
  214.         ##    Get-Content Delimited.csv | Convert-Delimiter "," "`t" | Set-Content Delimited.tab
  215.         ##         would convert a simple comma-separated-values file to a tab-delimited file
  216.         ################################################################################
  217.         ## Version History
  218.         ## Version 1.0
  219.         ##        First working version
  220.         ## Version 2.0
  221.         ##    Fixed the quoting so it adds quotes in case they're neeeded
  222.         ## Version 2.1
  223.         ##    Remove quotes which aren't needed
  224.         ## Version 2.2
  225.         ##    Trim spaces off the ends, they confuse things
  226.         ## Version 2.3
  227.         ##    Allow empty columns: as in: there,are,six,"comma, delimited",,columns
  228.         ## Version 2.3
  229.         ##    Replaced Trim() with regex to do similar job.
  230.         ##    if a parameter is "" <-- empty string this captures the "Quotes" as its value not empty string ???
  231. Function Convert-Delimiter([regex]$from,[string]$to)
  232. {
  233.    begin
  234.    {
  235.           $z = [char](222)
  236.    }
  237.    process
  238.    {
  239.           #$_ = $_.Trim()       # converted Trim into regex replace as powershell 1 does not have it ?
  240.           $_ = $_ -replace "^\s+", "" -replace "\s+$", ""
  241.           $_ = $_ -replace "(?:`"((?:(?:[^`"]|`"`"))+)(?:`"$from|`"`$))|(?:$from)|(?:((?:.(?!$from))*.)(?:$from|`$))","$z`$1`$2$z$to"
  242.           $_ = $_ -replace "$z(?:$to|$z)?`$","$z"
  243.           $_ = $_ -replace "`"`"","`"" -replace "`"","`"`""
  244.           $_ = $_ -replace "$z((?:[^$z`"](?!$to))+)$z($to|`$)","`$1`$2"
  245.           $_ = $_ -replace "$z","`"" -replace "$z","`""
  246.           $_
  247.    }
  248. }
  249.  
  250. # Use source code control client [svn] to retrieve referenced file.
  251. Function get-reference([string]$scc, [string]$filePath, [string]$rev, [string]$post, [string]$whiteSpacePre)
  252. {
  253.         $svn = 'svn.exe'
  254.  
  255.         if ($rev -eq $null -or $rev -eq "")
  256.         {
  257.                 # figure out the revision of head if we dont have it.
  258.                 $result = @(& $svn info $scc$filePath)  # ensure its an array even if only one
  259.                 $revResult = @($result -like "Revision: *")
  260.                 if ($revResult.Length -eq 1)
  261.                 {
  262.                         $tmp = $revResult[0]
  263.                         $regexRevision = [regex]"^Revision: (\d+)$"
  264.                         $myMatches = $regexRevision.match($tmp)
  265.                         if ($myMatches.Success)
  266.                         {
  267.                                 $rev = $myMatches.Groups[1]
  268.                         }
  269.                 }
  270.                 else
  271.                 {
  272.                         write-error "Cannot find revision for $scc$filePath."
  273.                         exit
  274.                 }
  275.         }
  276.        
  277.         # example of svn 'svn -r 1234 cat svn://scc/project/trunk/database/DBName/Procedure/procedure.sql'
  278.         write-host "Reference [$rev] $filePath"
  279.         #write-host "$svn -r $rev cat $scc$filePath"
  280.         $result = & $svn -r $rev cat $scc$filePath
  281.          
  282.         # prefix each line
  283.         for ($i = 0; $i -lt $result.Length; $i++)
  284.         {
  285.                 $result[$i] =  $whiteSpacePre + $result[$i]
  286.         }
  287.        
  288.         $fileContent = [string]::join("`r`n", $result)
  289.        
  290.         append-outputString "$comment ** Start Reference [$scc] [$filePath] [$rev] [$post]`r`n";
  291.         append-outputString "$fileContent`r`n"                          # additional newline after data
  292.         if ($post -ne $null -and $post -ne "")
  293.         {
  294.                 $convertedPost = $post -replace "\\n", "`r`n"  -replace "\\t", "`t"
  295.                 append-outputString $convertedPost
  296.         }
  297.         append-outputString "$comment ** End Reference [$scc] [$filePath] [$rev] [$post]`r`n";
  298. }
  299.  
  300. # procedures to wrap up global string buffer for processed output.
  301. function init-outputString()
  302. {
  303.         $global:outputString = ""
  304.         $global:sourceControl = ""
  305.         $global:outputFile = ""
  306. }
  307. function append-outputString([string]$str)
  308. {
  309.         $global:outputString += $str
  310. }
  311. function write-outputString([string]$file)
  312. {
  313.         write-output $global:outputString | out-file -filePath $file -encoding oem
  314. }
  315.  
  316. function execute-directive($splitDirective, [string]$whiteSpacePre)
  317. {
  318.         $keyword = $splitDirective[0]
  319.         switch ($keyword)
  320.         {
  321.                 "SourceControl"
  322.                 {
  323.                         write-host  "$comment $keyword `"$($splitDirective[1])`""
  324.                         append-outputString "$whiteSpacePre$comment $keyword `"$($splitDirective[1])`""
  325.                         $global:sourceControl = $splitDirective[1]
  326.                 }
  327.                 "OutputFile"   
  328.                 {
  329.                         append-outputString "$whiteSpacePre$comment $keyword `"$($splitDirective[1])`""
  330.                         $global:outputFile = $splitDirective[1]
  331.                 }
  332.                 "Reference"
  333.                 {
  334.                         get-reference $global:sourceControl $splitDirective[1] $splitDirective[2] $splitDirective[3] $whiteSpacePre
  335.                 }
  336.                 default
  337.                 {
  338.                         write-error "Unknown directive `"$keyword`" exiting..."
  339.                         exit
  340.                 }
  341.         }
  342. }
  343.  
  344. function process-directive([string]$directive, [string]$whiteSpacePre)
  345. {
  346.         $reDelimit = $directive | Convert-Delimiter " " "!"
  347.         $splitDirective = $reDelimit.Split("!")
  348.        
  349.         for ($i = 0; $i -lt $splitDirective.Length; $i++)
  350.         {
  351.                 $tmp = $splitDirective[$i]
  352.                 if ($tmp -eq "`"`"`"`"")                # convert """" to empty string -- side effect of Convert-Delimiter
  353.                 {
  354.                         $splitDirective[$i] = "";               # empty string
  355.                 }
  356.         }
  357.         execute-directive $splitDirective $whiteSpacePre
  358. }
  359.  
  360. function process-file([string]$inputFile, [string]$outputFile)
  361. {
  362.         init-outputString
  363.         $global:outputFile = $outputFile
  364.        
  365.         if (!(test-path -pathType Leaf $inputFile))
  366.         {
  367.                 write-error "Cannot open input file `"$inputFile`""
  368.                 exit
  369.         }
  370.        
  371.         $outmsg = "$comment ScriptSVN $version processing file `"$inputFile`""
  372.         write-host $outmsg
  373.         append-outputString "$outmsg`r`n"
  374.        
  375.         $inputLines = @(get-content -path $inputFile)   # read file @ to ensure we get an array
  376.         foreach ($line in $inputLines)
  377.         {
  378.                 $activeStr, $commentStr = split-comment $line
  379.                 $whiteSpacePre, $directive, $poststr = split-directive $activeStr
  380.                 if ($directive -eq $null -or $directive.Length -eq 0)
  381.                 {
  382.                         append-outputString "$activeStr"        # no directive to just output the line
  383.                 }
  384.                 else
  385.                 {
  386.                         process-directive $directive $whiteSpacePre
  387.                 }
  388.        
  389.                 # output comment
  390.                 if ($commentStr.Length -gt 0)
  391.                 {
  392.                         # convert input comment format to output format
  393.                         $afterComment = $commentStr.Substring($commentStart.Length)
  394.                         append-outputString "$comment$afterComment`r`n"
  395.                 }
  396.                 else
  397.                 {
  398.                         append-outputString "`r`n" # just new line
  399.                 }
  400.         }
  401.         write-host  "$comment OutputFile `"$($global:outputFile)`""
  402.         write-outputString $global:outputFile
  403. }
  404.  
  405. # by default process all file ending in file extension in current directory
  406. $files = @(get-childitem "*.dro")
  407.  
  408. if ($files.Length -eq 0)
  409. {
  410.         write-host "No files found for processing."
  411.         write-host "This utility processes files ending in `".dro`" "
  412. }
  413.  
  414. foreach ($f in $files)
  415. {
  416.         $outFile = $f -replace ".dro",".out"
  417.         process-file $f $outFile
  418. }

Submit a correction or amendment below (
click here to make a fresh posting)
After submitting an amendment, you'll be able to view the differences between the old and new posts easily.

Syntax highlighting:


Remember me