Friday, March 1, 2013

TFS Mirroring failover Checklist


In a mirrored TFS environment, there are quite a few items to verify after the failover from the primary site to the mirrored site. Here is the checklist.


  • Verify latest code. Before failover, make some changes. Verify the change after failover.
  • Verify changesets. Before failover, make some changes to create a new changeset. Verify the changeset after failover. May need to refresh a couple of time to get the latest changeset displayed.
  • Verify build definition and build history is also up-to-date      
  • Check in new code. Verify TFS functionality.
  • Verify build configuration from TFS admin console on the build machine. build service, build controller and build agent should all be up and running. May need to unregister the build service and create a new to connect to the new TFS server. For the build agent on the old primary TFS server to function, the TFS on the old server need to be started if not (%TFSInstallRoot%\tools\TFSServicecontrol.exe unquiesce). If want to use build controller on the new primary server, may need to configure build configuration from the new server and create a new controller and agent.
  • Verify the status of build controller and build agent from client VS -> build. Status should be available. 
  • Start new build using whichever build agent.   
  • Verify TFS web access. http://newserver:8080/tfs. click on links to verify.
  • Verify reports. http://newserver/reports. Generate at least one report to verify.

Monday, August 13, 2012

Configure SQL Server Mirroring Using Private Network

This article covers the steps to change the SQL Server database mirroring to use private network. Two additional NICs are installed on the principal and mirror TFS servers, connecting using cross-over network cable. The IP addresses are configured 10.10.10.1 and 10.10.10.2 respectively.

The list of databases to configure:
• ReportServer
• ReportServerTempDB
• STS_Content_TFS
• Tfs_Configuration
• Tfs_DefaultCollection
• Tfs_Warehouse

Step 1. Stop database alerts for both servers if any
SQL Server management studio -> SQL Server Agents -> Alerts -> disable all.

Step 2. Stop mirror job email alert for both servers if any
SQL Server management studio -> SQL Server Agents -> Jobs -> Database Mirroring Monitor Job -> Notification -> uncheck email.

Step 3. Backup the Database to be mirrored.
-- Backup the all databases on the Principal Server
use [master]
go

backup database [ReportServer]
to disk = 'F:\Backups\TFS1\ReportServer_mirror.bak';
go
backup log [ReportServer]
to disk = 'F:\Backups\TFS1\ReportServer_Log-mirror.trn';
go

backup database [ReportServerTempDB]
to disk='f:\backups\tfs1\ReportServerTempDB_mirror.bak';
go
backup log [ReportServerTempDB]
to disk='f:\backups\tfs1\ReportServerTempDB_Log-Mirror.trn';
go

backup database [STS_Content_TFS]
to disk='f:\backups\tfs1\STS_Content_TFS_mirror.bak';
go
backup log [STS_Content_TFS]
to disk='f:\backups\tfs1\STS_Content_TFS_Log-Mirror.trn';
go

backup database [Tfs_Configuration]
to disk='f:\backups\tfs1\Tfs_Configuration_mirror.bak';
go
backup log [Tfs_Configuration]
to disk='f:\backups\tfsfulldbbackups\tfs1\Tfs_Configuration_Log-Mirror.trn';
go

backup database [Tfs_DefaultCollection]
to disk='f:\backups\tfsfulldbbackups\tfs1\Tfs_DefaultCollection_mirror.bak';
go
backup log [Tfs_DefaultCollection]
to disk='f:\backups\tfsfulldbbackups\tfs1\Tfs_DefaultCollection_Log-Mirror.trn';
go

backup database [Tfs_Warehouse]
to disk='f:\backups\tfsfulldbbackups\tfs1\Tfs_Warehouse_mirror.bak';
go
backup log [Tfs_Warehouse]
to disk='f:\backups\tfsfulldbbackups\tfs1\Tfs_Warehouse_Log-Mirror.trn';
go

Step 4. Remove mirroring for every database from Principal server.
SQL Server Management Studio -> databases -> select a database -> properties -> mirroring -> remove mirroring

Step 5. Change endpoints on principal server.
-- alter Database Mirroring Endpoint on Principal Server
alter ENDPOINT [Mirroring]
STATE=STARTED
AS TCP (LISTENER_PORT = 5022, LISTENER_IP = (10.10.10.1))
FOR DATA_MIRRORING (ROLE = PARTNER, AUTHENTICATION = WINDOWS NEGOTIATE,
ENCRYPTION = REQUIRED ALGORITHM RC4);
GO

GRANT CONNECT ON ENDPOINT::Mirroring TO [ad\MYSERVICEACCOUNT];
GO

Step 6. Change endpoints on mirror server.
-- alter Database Mirroring Endpoint on Mirror Server
alter ENDPOINT [Mirroring]
STATE=STARTED
AS TCP (LISTENER_PORT = 5022, LISTENER_IP = (10.10.10.2))
FOR DATA_MIRRORING (ROLE = PARTNER, AUTHENTICATION = WINDOWS NEGOTIATE,
ENCRYPTION = REQUIRED ALGORITHM RC4);
GO

GRANT CONNECT ON ENDPOINT::Mirroring TO [ad\MYSERVICEACCOUNT];
GO

Step 7. Verify endpoints from both servers
-- Verify the Database Mirroring Endpoint Status
SELECT name, protocol_desc, state_desc FROM sys.database_mirroring_endpoints
GO

Step 8. Copy the back up files from TFS1 to TFS2, using cross-over interfaces

Step 9. Restore the all databases on the Mirrored instance using NORECOVERY option. make sure the TFS_Configuration is the first one to restore
-- Restoring the database ReportServer from the backup file
USE [master]
GO
alter database [TFS_Configuration] set partner off

RESTORE DATABASE [TFS_Configuration]
FROM DISK = 'F:\TFS1_Backups\TFS_Configuration_mirror.bak'
WITH REPLACE, NORECOVERY;
GO
RESTORE LOG [TFS_Configuration]
FROM DISK = 'F:\TFS1_Backups\TFS_Configuration_Log-mirror.trn' WITH NORECOVERY;
GO

USE [master]
GO
alter database [ReportServer] set partner off

RESTORE DATABASE [ReportServer]
FROM DISK = 'F:\TFS1_Backups\ReportServer_mirror.bak'
WITH REPLACE, NORECOVERY;
GO
RESTORE LOG [ReportServer]
FROM DISK = 'F:\TFS1_Backups\ReportServer_Log-mirror.trn' WITH NORECOVERY;
GO

USE [master]
GO
alter database [ReportServerTempDB] set partner off

RESTORE DATABASE [ReportServerTempDB]
FROM DISK = 'F:\TFS1_Backups\ReportServerTempDB_mirror.bak'
WITH REPLACE, NORECOVERY;
GO
RESTORE LOG [ReportServerTempDB]
FROM DISK = 'F:\TFS1_Backups\ReportServerTempDB_Log-mirror.trn' WITH NORECOVERY;
GO

USE [master]
GO
alter database [STS_Content_TFS] set partner off

RESTORE DATABASE [STS_Content_TFS]
FROM DISK = 'F:\TFS1_Backups\STS_Content_TFS_mirror.bak'
WITH REPLACE, NORECOVERY;
GO
RESTORE LOG [STS_Content_TFS]
FROM DISK = 'F:\TFS1_Backups\STS_Content_TFS_Log-mirror.trn' WITH NORECOVERY;
GO

USE [master]
GO
alter database [TFS_DefaultCollection] set partner off

RESTORE DATABASE [TFS_DefaultCollection]
FROM DISK = 'F:\TFS1_Backups\TFS_DefaultCollection_mirror.bak'
WITH REPLACE, NORECOVERY;
GO
RESTORE LOG [TFS_DefaultCollection]
FROM DISK = 'F:\TFS1_Backups\TFS_DefaultCollection_Log-mirror.trn' WITH NORECOVERY;
GO

USE [master]
GO
alter database [TFS_Warehouse] set partner off

RESTORE DATABASE [TFS_Warehouse]
FROM DISK = 'F:\TFS1_Backups\TFS_Warehouse_mirror.bak'
WITH REPLACE, NORECOVERY;
GO
RESTORE LOG [TFS_Warehouse]
FROM DISK = 'F:\TFS1_Backups\TFS_Warehouse_Log-mirror.trn' WITH NORECOVERY;
GO

Step 10. Setup the Mirroring sessions on mirror server.
-- Adding the database to Database Mirroring Session (Execute it on Mirror Server)

USE [master]
ALTER DATABASE [ReportServer]
SET PARTNER = 'TCP://10.10.10.1:5022';
GO

USE [master]
ALTER DATABASE [ReportServerTempDB]
SET PARTNER = 'TCP://10.10.10.1:5022';
GO

USE [master]
ALTER DATABASE [STS_Content_TFS]
SET PARTNER = 'TCP://10.10.10.1:5022';
GO

USE [master]
ALTER DATABASE [Tfs_Configuration]
SET PARTNER = 'TCP://10.10.10.1:5022';
GO

USE [master]
ALTER DATABASE [Tfs_DefaultCollection]
SET PARTNER = 'TCP://10.10.10.1:5022';
GO

USE [master]
ALTER DATABASE [Tfs_Warehouse]
SET PARTNER = 'TCP://10.10.10.1:5022';
GO

Step 11. Setup the Mirroring sessions on principal server.
-- Adding the database to Database Mirroring Session (Execute it on Principal Server)
USE [master]
GO
ALTER DATABASE ReportServer SET PARTNER OFF;
ALTER DATABASE ReportServerTempDB SET PARTNER OFF;
ALTER DATABASE STS_Content_TFS SET PARTNER OFF;
ALTER DATABASE Tfs_Configuration SET PARTNER OFF;
ALTER DATABASE Tfs_DefaultCollection SET PARTNER OFF;
ALTER DATABASE Tfs_Warehouse SET PARTNER OFF;

USE [master]
ALTER DATABASE [ReportServer]
SET PARTNER = 'TCP://10.10.10.2:5022';
GO

USE [master]
ALTER DATABASE [ReportServerTempDB]
SET PARTNER = 'TCP://10.10.10.2:5022';
GO

USE [master]
ALTER DATABASE [STS_Content_TFS]
SET PARTNER = 'TCP://10.10.10.2:5022';
GO

USE [master]
ALTER DATABASE [Tfs_Configuration]
SET PARTNER = 'TCP://10.10.10.2:5022';
GO

USE [master]
ALTER DATABASE [Tfs_DefaultCollection]
SET PARTNER = 'TCP://10.10.10.2:5022';
GO

USE [master]
ALTER DATABASE [Tfs_Warehouse]
SET PARTNER = 'TCP://10.10.10.2:5022';
GO

Step 12. On the Database Mirroring Properties change the Operation Mode to High Performance (asynchronous) for every database.
SQL Server Management Studio -> databases -> select a database -> properties -> mirroring -> Operation mode -> change to High Performance (asynchronous)

Step 13. Verify mirroring setting from principal server
SQL Server Management Studio -> databases -> select a database -> properties -> mirroring

Step 14. Verify mirroring by checking database status from both servers
SQL Server Management Studio -> databases

Step 15. Enable database alerts for both servers
SQL Server management studio -> SQL Server Agents -> Alerts -> enable all.

Step 16. Start mirror job email alert for both servers
SQL Server management studio -> SQL Server Agents -> Jobs -> Database Mirroring Monitor Job -> Notification -> check email.

Wednesday, February 22, 2012

Change Shelveset owner in TFS 2010

Inside TFS database schema, shelvesets are saved in tbl_workspace table with type=1.
use Tfs_defaultcollection;
select * from tbl_Workspace where type = 1 ;

One of the ways to get ownerID is from a developer's workspace name.
use Tfs_defaultcollection;
select * from tbl_Workspace where WorkspaceName='MYWORKSPACENAME';

To change the owner of a shelveset, run the following command:
use Tfs_defaultcollection;
update tbl_Workspace set ownerid=5 where type = 1 and WorkspaceName='shelvetest_3';

This may be useful after a develop leaves the team and his Active Directory ID is revoked.

Note: It is highly recommended not to modify TFS database directly.

Thursday, January 5, 2012

Revoke Read Access to ClearCase VOB

Say a VOB (VOB tag: VOB1) only allows a group of developers (group id: GRPa) to have read-write access. All other users do not have any access.

1. Make sure that GRPa is either listed as ownership group or additional group. This privilege defines who can modify the code.

> cleartool desc -l vob:VOB1
...
VOB ownership:
owner AD\ccadmin
group AD\clearcase
Additional groups:
group AD\GRPa
...

To add GRPa to the addiontinal group, run command
> clearcase protectvob -add_group GRPa

2. Make sure that the VOB root directory has 770 and owned by GRPa group. This privilege defines who can see the code under VOB root.

> cleartool desc -l VOB1
...
Element Protection:
User : ccadmin : rwx
Group: GRPa : rwx
Other: : ---
...

to change the protection of the VOB root directory, run command

> cleartool protect -chmod 770 VOB1


If another group (group id: GRPb) needs only read access to VOB1, then a third group (id GRPc) needs to be created to include both GRPa and GRPb. The group GRPc need to be the group of VOB root element. To change it, run command

> cleartool protect -chgrp GRPc VOB1

How to run cleartool mkattr from ant script

Managing escape charactor is never a simple task, especially when it involves xml, cleartool, ant exec, and trying to publish a piece of html code on a blog. Here is an example of how to set an attribute to a ClearCase baseline.

<!-- set baseline with the new build label-->
<exec dir="${clearcase.bin}" executable="cleartool" failonerror="true" >
   <arg line="mkattr"/>
   <arg line="BuildNO"/>
   <arg line="&apos;\&quot;${build.label}\&quot;&apos;"/>
   <arg line="${cc.baseline.new}"/>
</exec>

Thursday, October 27, 2011

Delete DLL failed due to "required by other applications" Error

Sometimes, when we try to delete a DLL, it fails due to "required by other applications" error.

Normally, we can delete a DLL from GAC by gacutil.exe tool shipped with .NET 1.1, or use the newer interface, the Assembly Cache Viewer,  integrated into Windows Explorer, located at %windir%\assembly.

For the assembly initially is installed using msi package, try to use the setup msi to uninstall the DLL.

For msi installed assembly, as the last resort, back up following registry key HKLM\Software\Classes\Installer\Assemblies\Global\. Delete the corresponding entry for the pinned DLL. (for per-machine installations, use HKCU\SOFTWARE\Classes\Installer\Assemblies\Global). Then go back to the GAC to delete the DLL.

Friday, September 30, 2011

How to find out SMTP server

In command prompt, type nslookup and hit enter
set type=MX

Type the domain name and hit enter, for example
> AD

The results will be a list of host names that are set up for SMTP
Server:  xxx.xxx.com
Address:  xxx.xxx.xxx.xxx

or, under nslookup prompt, type
> set q=mx 
> mailhost

Tuesday, August 24, 2010

ClearCase Deliver with "unable to compute base" Error

Problem Symptoms:

1. Rebase and deliver operation could not finish on one particular file. Error message:
>>> Operation started: 8/17/2010 3:29:45 PM
Error from VOB database: "\MidTier".
Element "xxxx\InquiriesRequestVOFactory.java", unable to compute base (to \main\Mainline_i\CBFE17.5_THS\djogo_CBFE17.5_THS\29 from \main\Mainline_i\CBFE17.5_THS\36).
Skipping "xxxx\InquiriesRequestVOFactory.java".

Other related errors.
1. The version tree of the file could not be retrieved from GUI with the following error message. But cleartool lsvtree command ran without error.
"failed to retrieve version history of the element"

2. db_server_log error message.
% more db_server_log
2010-08-16T09:17:10-04:00 db_server(15416): Ok:
2010-08-17T12:47:07-04:00 db_server(11335): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T12:55:13-04:00 db_server(11335): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T12:57:38-04:00 db_server(11336): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T13:02:54-04:00 db_server(12862): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T13:06:08-04:00 db_server(12862): Error: Database identifier 724419 not found in "../db__obj.c" line 740.
2010-08-17T13:06:49-04:00 db_server(12862): Error: Database identifier 724419 not found in "../db__obj.c" line 740.

3. could not merge the file.
>cleartool merge -to InquiriesRequestVOFactory.java -version \main\Mainline_i\CBFE17.5_THS\36
cleartool: Error: Error from VOB database: "\MidTier".

But the file could be merge with only drawing merge arrows.
>cleartool merge -to InquiriesRequestVOFactory.java -ndata -version \main\Mainline_i\CBFE17.5_THS\36
Recorded merge of "InquiriesRequestVOFactory.java".

Workaround.
By drawing an merge arrow as above, we can skip the element for deliver/rebase. but it doesnot resolve the problem.

Diagnose.
1. run command dbcheck.
/etc/utils/dbcheck -r1 -a -k -p32767 vob_db
...
Processing data file: vob_db.d02(3), total of 1302988 records
Problems at record 1298822:
* key field OBJ_DBID(23) error: has a missing key
...
1 error was encountered in 1 record/node

Solution.
1. Stop ClearCase (or untag and unregister the VOB, stop ClearCase and then start ClearCase)
2. Backup (very import!). Copy the db directory
3. Copy the keybuild utility to the VOB db directory
4. run: keybuild vob_db under the VOB db directory
5. Exit out of the db directory after keybuild completes
6. Start ClearCase
7. Reformat the VOB. Run the following command:
cleartool reformatvob

If these steps are completed successfully and without errors, your VOB should now be healthy. If they did not complete or you received errors, restore the VOB from backup.

For my VOB with 7.4GB in size and 5 years in history, it took about 3 hours to finish the procedure.

Thursday, June 10, 2010

Report Server rsInternalError Error

When accessing all the TFS reports from report server, the following message showed up in browser.
An internal error occurred on the report server. See the error log for more details. (rsInternalError)

Further investigation found more information in the Reporting Service log file and dump log file.

ERROR , SQLDUMPER_UNKNOWN_APP.EXE, AdjustTokenPrivileges () failed (00000514)

w3wp!processing!5!06/07/2010-08:48:03:: a ASSERT: Assertion failed! Call stack:
Microsoft.ReportingServices.ReportProcessing.Persistence.IntermediateFormatReader.Assert(Boolean condition)
Microsoft.ReportingServices.ReportProcessing.Persistence.IntermediateFormatReader.Initialize(Stream stream)
Microsoft.ReportingServices.ReportProcessing.Persistence.IntermediateFormatReader..ctor(Stream stream)
Microsoft.ReportingServices.ReportProcessing.ReportProcessing.DeserializeReport(GetReportChunk getChunkCallback, ReportItem parent, Hashtable& definitionObjects)

These reports used to work, and there was no known changes to the system. I do notice that the size of chunkData table in the reportserver database became 0 around the same time the problem started.

To solve the problem, we need to restore the reports. From SQL Server Management Studio, run

use reportserver
go

SELECT
itemID, Path, Name, CAST(CAST(Content AS varbinary(max)) AS xml) ReportXML
FROM
dbo.Catalog
where content is not null

The ReportXML column is the report definition. Save it to a name.rdl file, and upload the file to the report site to overwrite the report.

This can not be the best solution, as I have 900+ reports to recover. But it is the only working one I found so far.

Wednesday, June 2, 2010

Hoxfix KB957196 Install procedure for database-tier mirrored TFS sites

This procedure also applies to the installation of TFS upgrades SP1 on database-tier mirrored sites.

1. before start, notify the users and make sure nobody is using TFS.
2. backup Report Server encryption key. Ship the key to site2. The key will not be used in the procedure. It will be used if the primary site needs to recover.
3. open SQL Server Management Studio, connect to site1 and site2.
4. in Management Studio, full database + transaction backup for site1. Ship the backups to site2.
6. in Management Studio, stop mirroring from site1.
7. install hotfix for 750-smart-bld1.
8. backup full database + transaction log. TFSIntegration database is modified during hotfix install.
9. ship the backup set to site2.
10. in Management Studio, restore database (and transaction log) to open status for site2.
11. in Management Studio, set up mirroring from site1.
13. full backup again for site1.

Monday, May 31, 2010

Mostly Recently Updated Table for SQL Server

I found a useful query for finding out the most recently accessed/updated table in a SQL Server database (I forgot the original resource). I used the script to find out a table back sceen of Team Foundation Server (TFS), so I could modify the content to configure a particular URL.

use TfsIntegration     
go
select
t.name
,last_user_update
,user_updates
,user_seeks
,user_scans
,user_lookups
,last_user_seek
,last_user_scan
,last_user_lookup
from
sys.dm_db_index_usage_stats i JOIN
sys.tables t ON (t.object_id = i.object_id)
where
database_id = db_id()

SQL Server database hang in "in recovery" status

Problem. After a failed setting mirror operation (initiated from SQL Server Management Studio from GUI), SQL Server Database stay in "in recovery" status for prolonged period of time (overnight). The database is about 7GB in size. From the primary site, the database showed normal. From the secondary site, the database showed "in recovery".

There is little that can be done for the secondary database. One option is to stop the SQL services, rename the data files (.mdf & .ldf), start the SQL database again, drop the existing database, rename the data files back, and attached the data files with same database name. After that, we can try to restore database backup and set up mirror.

On a closer look from primary, it seems that the mirror is broken. However, from the output of some database operation in the secondary database, it seems that the mirror has not be broken yet. Further more, there is no entry in the event log to indicate the progress of the recovery operation.

Let's try this approach. From the primary site, to break the mirror run the following command.
ALTER DATABASE TfsVersionControl SET PARTNER OFF
After the command finished successfully, the database on the secondary site changed to (Restoring…) status immediately. The following operations are straight forward -- copy the full database backup and log, and restore with norecovery option, set mirror for the database.
This is the preferred approach, since only the problem db is touched, no need to re-set the mirror for all other databases.

There are occasional complains from the internet about this prolonged (or hanged) SQL Server database "in recovery" status. It may worth to take a look whether the database actually started the "recovery", or still in a previous status due to unknown reasons.

Script to Monitor ClearCase license

Here are the scripts mentioned in my previous post regarding monitoring ClearCase license usage.

re: http://doublepaddle.blogspot.com/2008/02/monitoring-clearcase-license-usage.html Apperently I am not good at following up my previous posts -- Sorry for the delay.


The collection script is as simple as this.

#!/usr/bin/sh
date >> /home/ccadmin/logs/license.log
/opt/rational/clearcase/bin/clearlicensegrep "Current active users" >> /home/ccadmin/logs/license.log

The script is scheduled in cron jobs for every 10 mintues. This script can be easily converted to a Windows batch file and added to scheduler.

Then a perl script is used to load the data and generate the report in excel format. The script can be added as part of the Windows scheduler, or run at the time when you want to view the report. The nice chart in the previous post is also part of the excel report.

#!/usr/bin/perl
#Auther: Li Qin
#Date: 2008.02.05
#Function: generate ClearCase license usage report in excel format

use Spreadsheet::WriteExcel;
use Win32::OLE qw(in with);
use Win32::OLE::Const 'Microsoft Excel';

$Win32::OLE::Warn = 3; # die on errors...

$file = "C:\\Li\\SCM\\ClearCase\\report-licns.xls";
$workbook = Spreadsheet::WriteExcel->new("$file");
$sheet = $workbook->addworksheet("data");

my $format1 = $workbook->add_format();
$format1->set_bold();
$format1->set_size(15);
$format1->set_color('blue');
$format1->set_align('center');

my $format2 = $workbook->add_format();
$format2->set_bold();
$format2->set_size(11);
$format2->set_color('Black');
$format2->set_align('center');

my $format3 = $workbook->add_format();
$format3->set_size(11);
$format3->set_color('Black');
$format3->set_align('center');

$sheet->activate();
$sheet->merge_range('C2:G2', "ClearCase Usage Report", $format1);

$sheet->set_column(3, 2, 40);
$sheet->set_column(3, 3, 20);
$sheet->set_column(3, 4, 20);
$sheet->write(3, 2, "Date", $format2 );
$sheet->write(3, 3, "Usage", $format2 );
$sheet->write(3, 4, "Max", $format2 );

open(FILE,"license.log")  || die "couldn't open file a.out";
$row = 4;
my @user1, @user2;
$max = 35;
while ($date= ) {
chomp $date;

$line1 = ;
chomp $line1;
while ( !($line1 =~ m/Current/) ) {
$user1[$row-4] = 0;

$sheet->write($row+1, 2, $date, $format3);
$sheet->write($row+1, 3, $user1[$row-4], $format3);
if ( $date =~ m/Fri Feb 1 17/ ) {
$max = 40;
}
$sheet->write($row+1, 4, $max, $format3);
$row = $row +1;
$date = $line1;
$line1 = ;
chomp $line1;
}
$pos = rindex $line1, ":";
$user1[$row-4] = substr($line1,$pos+1);
$user1[$row-4]=$user1[$row-4]+0;

$sheet->write($row+1, 2, $date, $format3);
if ( $date =~ m/Fri Feb 1 17/ ) {
$max = 40;
}
$sheet->write($row+1, 3, $user1[$row-4], $format3);
$sheet->write($row+1, 4, $max, $format3);
$row = $row +1;
}
close(FILE);
$workbook->close;

# #create chart
my $Excel = Win32::OLE->GetActiveObject('Excel.Application') || Win32::OLE->new('Excel.Application', 'Quit');

# open Excel file
my $Book = $Excel->Workbooks->Open("$file");
# select worksheet of data


$count = 6;
my $sheet = $Book->Worksheets("data");
my $value = $sheet->Cells($count,4)->{'Value'};
my $len = length($value);

while ( $len >0 ) {
$count = $count +1;
my $value = $sheet->Cells($count,4)->{'Value'};
$len = length($value);
#printf "At ($count, 4) the value is %s. length is %d\n", $value, $len;
}
print "last row is $count.";


$lastRow = $count-1;
my $Range = $sheet->Range("C6:E$lastRow");

my $Chart2 = $Excel->Charts->Add($sheet);
$Chart2->{ChartType} = xlLine;
$Chart2->SetSourceData({Source => $Range, PlotBy => xlColumns});

$Chart2->{HasTitle} = 1;
$Chart2->ChartTitle->{Text} = "ClearCase License Usage\n";
$Chart2->Axes(xlValue)->{HasTitle} = 1;
$Chart2->Axes(xlValue)->AxisTitle->{Text} = "License Usage";
$Chart2->Axes(xlCategory)->{HasTitle} = 1;
$Chart2->Axes(xlCategory)->AxisTitle->{Text} = "Date/Time";

$Chart2->{HasLegend} = 'False';
$Chart2->{Name} = "Chart";
$Chart2->{HasLegend} = 1;
$Chart2->Legend->{Position} = xlBottom;
$Chart2->SeriesCollection(1)->{Name} = "Usage";
$Chart2->SeriesCollection(2)->{Name} = "Max";

# save and clean up
$Book->Save;
$Book->Close;
# End of create charts

We have upgraded our monitoring using SQL Server Reporting Service, a nice-to-have.

Monday, May 17, 2010

Change check-in email notification to TFS Web Access

There are some very useful articles to change TFS email notifications to link to Team System Web Access:

Changing TFS emails to link to Team System Web
Changing TFS emails to link to Team System Web Part 2

The configurable links include:
- Work Item notification mails
- Build notification mails
- Check-in notification mails
- Mails sent by Team Explorer

However, for check-in notification mails, if your TFS web access is not installed as the default website, and contains a virtual directory as part of the URL, the tfsadminutil tool cannot process the URL correctly, at least not until TFS 2008 SP1. For example, if your TFS web access URL is http://myTFSserver:8080/tswa, tfsadminutil tool will set up the links as http://myTFSserver:8080/ instead.

Here we provide a workaround until tfsadminutil tool has that improved. The workaround requires direct modification of the TFS database. Please note that it is NOT recommended by Microsoft.

Prerequisite: TFS 2008 + SP1 + TFS web access + Hotfix KB957196

Step 1. run the command on the TFS server:
C:\Program Files\Microsoft Visual Studio 2008 Team Foundation Server\Tools> tfsadminutil configureconnections /TSWAUri:http://myTFSserver:8080/tswa

Step 2. connect to TFS database through SQL Server Management Studio. Run the following query.
use TfsIntegration
go
select * from tbl_service_interface

output:
....
8 WorkItemEditor
http://myTFSserver:8080/wi.aspx
8 ChangesetDetail
http://myTFSserver:8080/cs.aspx
8 Difference
http://myTFSserver:8080/diff.aspx
8 ViewItem
http://myTFSserver:8080/view.aspx
....

Step 3. Modify the URLs in the table for the above items.
Update tbl_service_interface
Set url='http://myTFSserver:8080/tswa/wi.aspx'
Where name='WorkItemEditor'

Update tbl_service_interface
Set url='http://myTFSserver:8080/tswa/cs.aspx'
Where name='ChangesetDetail'

Update tbl_service_interface
Set url='http://myTFSserver:8080/tswa/diff.aspx'
Where name='Difference'

Update tbl_service_interface
Set url='http://myTFSserver:8080/tswa/view.aspx'
Where name='ViewItem'

Step 4. After the TFS process is recycled, your check-in email subscription will generate emails linking to TFS web access.

Thursday, February 4, 2010

No Views Show up in ClearCase Explorer

One developer got problem accessing his view. ClearCase used to work fine on his desktop. Since two days ago, after refreshing the ClearCase Explorer view shortcuts, he found that there were no views showing up on the view panel. He had no problem using ClearCase Project Explorer, except for view related operations, such as list the view property.








When trying to start the view manually, it showed the following error.

H:\>cleartool startview patelsus_VER6.1.5_int
view_contact call failed: RPC: Unable to receive; errno = [WINSOCK] Connection reset by peer
cleartool: Error: Error trying to contact view_server for view Tot75xrdtw1027q:C:\CCShare\patelsus_VER6.1.5_int.vws: No such file or directory
cleartool: Error: Couldn't set view tag patelsus_VER6.1.5_int: No such file or directory


There were errors in the system's event log too, complaining view server died on start up, etc.

After rule out other causes, run winmsd from Start -> Run, under the network -> protocal, the first entry should be "MSAFD Tcpip [TCP/IP]" and the second should be "MSAFD Tcpip [UDP/IP]". However, the protocal showed: "SSH Capture over [MSAFD Tcpip [TCP/IP]]" and "SSH Capture over [MSAFD Tcpip [UDP/IP]]".


It turned out that the developer installed SSH Tectia, which conflicts with ClearCase client. After uninstallation, the problem is resolved.

Wednesday, December 9, 2009

Install SQL Server Bussiness Intellegent Development Studio

If you have SQL Server Management Studio Express, SQL Server Bussiness Intellegent Development Studio (BIDS) is not installed by default. But you can get BIDS as part of Microsoft SQL Server 2005 Express Edition Toolkit. Microsoft SQL Server 2005 Express Edition Toolkit can be freely downloaded from Microsoft.

Problem:

When installing Microsoft SQL Server 2005 Express Edition Toolkit, the installation failed with the following error message in the log file.

Machine : xxxxxx

Product : MSXML 6 Service Pack 2 (KB954459)

Product Version : 6.20.1099.0

Install : Failed Log File : c:\Program Files\Microsoft SQL Server\90\Setup Bootstrap\LOG\Files\SQLSetup0005_TOT75XRDTW1027R_MSXML6_1.log

Error Number : 1603

The detailed error message in the log indicates

Action start 10:34:08: SkipInstallCA.

This package is not supported on this operating system.

Action ended 10:34:08: SkipInstallCA. Return value 3.

Solution.

First uninstall the existing version of msxml. If fail to uninstall, use Windows Installer CleanUp Utility to clean up msxml6.

After that, re-run SQLEXPR_TOOLKIT.EXE to install BDIS. Everything should be fine.

Wednesday, October 7, 2009

TFS: Unable to connect to this Team Foundation Server

When using Visual Studio 2008 to connect to TFS server from a client machine, I got the following error.


Microsoft Visual Studio

TF31002: Unable to connect to this Team Foundation Server: mytfsserver.

Team Foundation Server Url: http://mytfsserver:8080.
Possible reasons for failure include:
- The Team Foundation Server name, port number or protocol is incorrect.

- The Team Foundation Server is offline.

- Password is expired or incorrect.


For further information, contact the Team Foundation Server administrator.


Further investigation shows that the same user can connect to the server from other machine, and other user can connect to the server from the same client machine. All the users and the server reside in the same domain. It must be the setting for that user on that client machine.

Because Team Explorer uses IE controls behind the scene, I opened the IE options and found that the LAN setting has configured to use proxy. After unchecked the setting, VS2008 has no problem connecting to the TFS server.

Monday, June 29, 2009

MSI: Uninstall left some files not removed

I created a MSI package and tested the installation and uninstallation. Everything looked good except for one file. The abc.exe file could not be removed by the uninstallation process. The INSTALLDIR started clean. And file was not used at the time of uninstallation.


I created the extensive log using /L*v option with msiexec command. After studying the log file and comparing the ComponentId in the msi, I found out that the file was marked "PreviouslyPinned".


MSI (s) (50:C0) [15:22:31:932]: Executing op: ComponentUnregister(ComponentId={4CA897DC-C1DD-1A1D-CA93-FF18DA884529},,BinaryType=0,PreviouslyPinned=1)1: {2E9A386D-9B96-4E7C-9AE9-B614A86EEFA5} 2: {4CA897DC-C1DD-1A1D-CA93-FF18DA884529}


Sure enough, I found an entry of the file in HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\SharedDLLs.

After deleting the entry, the uninstallation of the MSI worked properly. The MSI package was correct, the problem resided on the testing environment.

Friday, June 5, 2009

Configure Cygwin sshd for Build Automation

Our build is initiated from a UNIX build server, then invokes the Windows build portion.

1. create a local build user on the Windows build server. Use the same id as in the UNIX server. Password can be different.

2. install cygwin on the Windows build server.

3. set up sshd service.
There is a very nice article about how to set up sshd using cygwin, including which cygwin packages to install, how to install and configure ssh service, how to generate user pulib/private keys, and how to add the user to the password file.
In summary, use the following command.
ssh-host-config
ssh-user-config
You should be able to find a "CYGWIN sshd" service in Windows machine. Make sure the service is started.

4. in the ssh client (UNIX build server), login as build id, generate rsa (or dsa) public/private keys. entry cartridge return (empty) for passphrase.
# ssh-keygen -t dsa
public key is save to ~/.ssh/id_dsa.pub
private key is saved to ~/.ssh/id_dsa

5. sftp the public key(id_dsa.pub) to ~/.ssh directory on the ssh SERVER (Windows build server).

6. create an authorization file in the ~/.ssh directory on SERVER, add the public key in.

server> cat /id_dsa.pub >> /.ssh/authorized_keys
7. To verify that you can connect to the target system, log in through from the client. An entry will be created in the ~/.ssh/known_hosts file in the server.
$ ssh id@target
Now you can integrate the Windows build with UNIX build process.

...
ssh -n -o NumberOfPasswordPrompts=0 ccadmin@winbuild "rm -rf /cygdrive/e/CBFE_Build/${RELEASE}"
ssh ccadmin@winbuild "cd /cygdrive/e/CBFE_Build/${RELEASE}; /usr/bin/unzip -u /cygdrive/e/CBFE_Build/${FILENAME}"
ssh ccadmin@winbuild "cd /cygdrive/e;/cygdrive/c/Program\ Files/InstallShield/2009/System/IsCmdBld.exe -p CBFE_Build/IS_Projects/${RELEASE}/CBFE_Common/CBFE_Common.ism -z BUILD_VERSION=${BUILD_LABEL}"
....

Thursday, June 4, 2009

Run InstallShield ISCmdBld.exe under cygwin, ssh

After installed and configured cygwin (sshd) in the Windows build server, I tested the ssh connection without any issue. The next step is to launch the InstallShield command line builder ISCmdBld.exe through ssh from my UNIX build box.


ssh myUserid@myWindowsBuildServer "cd /cygdrive/e;/cygdrive/c/Program\ Files/InstallShield/2009/System/IsCmdBld.exe -p CBFE_Build/IS_Projects/myProject.ism -z BUILD_VERSION=${BUILD_LABEL}"


Nothing happened. The command prompt returned immediately. After more testing, I found out that if I logged in to the Windows build server using myUserid, or other id, I had no problem running ISCmdBld.exe in a cygwin session. However, logging in through ssh from other boxes (UNIX or Windows) did not work. It was not X server. And it was not the ACL on the executables. Even power user group is not sufficient.

Finaly, by adding the user to the Administrator group on the Windows box, the problem solved.