My view on technology

My Backup script for Mac

2011-06-21T12:00:00.000-07:00

Recently I bought an external disk of 1 terabyte. This comes to replace my old 250 MB disk.

The fists thing I made was to format the disk and create 1 partition to store all the information. As I wanted to use the disk with Mac and PC, I formatted it as NTFS. I also acquire a Mac utility to support writing to NTFS.

The disk worked fine until I tried to use backup tools for Mac. Most of the backup tools for Mac require a disk formatted in Mac format. But in my case it is NTFS. A solution might be to create 2 partitions, one for Mac and one for PC, but the one of Mac might not be available in PC. Still I want the compatibility with both systems.

My backup requirements for Mac were easy:

Back up only data files
Maintain only the image at the time the backup is made
Do not backup OS and Apps. In case of failure, those can be restored from the installation disk and packages.

I was about to format the disk with the 2 partitions when I realize I already had 500MB of information in the disk. I was in risk to lose part of my information because I have no other disk to use as backup during the format operation.

I include another requirement:

The backup files can be read from both systems. In case I want to review the files in any system.

The backup is simplified as I’m not really interested in maintain version of files.

I started looking for some scripts and backup ideas but none of those I found were of my entirely satisfaction.

At last I decide to create my own. The easiest seems to be the RSYNC command to create the copy of my Mac’s data.

My Backup Script initially was only to RSYNC the folder I want to backup, creating a copy of into the external disk.

The first time it took hours to backup all files.

The second time again took hours to make the backup.

No matter what files were changed the RSYNC always took hours to finish.

I didn’t do any deep investigation, but I’m convinced the RSYNC command can’t handle properly files stored under NTFS format. This because when the RSYNC started, many files are detected as changed, no matter no single change was made.

After been disappointed by RSYNC, I create my own script to sync files.

The required operation for the script:

Receive 2 folders, source and destination
Validate those folders
Compare current files to detect differences
Copy from source to destination the newly created files
Delete from destination the deleted files
Update destination if changes were made to source

Compare current files to detect differences, as needed: Copy from source to destination the newly created files Delete from destination the deleted files Update destination if changes were made to source

Using existing BASH commands all these can be done.

The script in review

The basic script usage is:

./myBackup.sh sourceDir destDir

./myBackup.sh sourceDir destDir X

Given the command, the script starts. We will later discuss the functions.

First validate the parameters

validateSource "$1"

...

validateDest "$2"

If source does not exist, the script won’t continue.

The list of files from both directories is retrieved using the FIND command.

cd "$source"

find -s * > $currentDir/__source__.lis

...

cd "$dest"

find -s * > $currentDir/__dest__.lis

Using the DIFF command we can get the differences.

diff __source__.lis __dest__.lis > __diff__.lis

Now, in the __diff__.lis file we have the list of differences. Reading that file we can process the fists 2 steps of the synchronization.

I had problems performing 2 operations as I read the file, so, I read the differences file twice, one for deletion, one to copy files.

First loop is to delete files. The files deleted from source are deleted from destination in this loop.

while read line

# < Copy file from source to dest

# > Delete file from dest

if [ "${line:0:1}" == ">" ]

then

delDest "$dest" "${line:2}"

done < __diff__.lis

Second loop is to copy files. The files created in source and absent in destination are copied in this loop.

while read line

# < Copy file from source to dest

# > Delete file from dest

if [ "${line:0:1}" == "<" ]

then

copySource2Dest "$source" "$dest" "${line:2}"

done < __diff__.lis

To detect differences in files, the evaluation is:

Compare changed date
Compare file size
If required, compare checksum

For this comparison, we loop the file list of source directory.

while read line

{process here}

done < __source__.lis

Within the loop, we get parameters for each file. Directories are skipped from this validation.

getFileProps "$source" "$line"

sourceCreate=$st_birthtime

sourceChange=$st_mtime

sourceSize=$st_size

getFileProps "$dest" "$line"

destCreate=$st_birthtime

destChange=$st_mtime

destSize=$st_size

Once we retrieve the parameters, the comparisons are very straight forward.

if [ $sourceChange -ne $destChange ]

then

different=1

if [ $sourceSize -ne $destSize ]

then

different=1

The script is prepared to receive 3 parameters; the third is any value and represents the need to compare checksums of each file (source and destination). The third parameter is stored in a variable $deepSearch

if [ ! $deepSearch == "" ]

then

set $(cksum "$source/$line")

sourceCheckSum=$1

set $(cksum "$dest/$line")

destCheckSum=$1

if [ $sourceCheckSum -ne $destCheckSum ]

then

different=1

If all the validations tells us the files are different, then, the file need to be copied.

if [ $different -eq 1 ]

then

needEnter=0

copySource2Dest "$source" "$dest" "$line"

Finally, just delete the temporary files.

rm __source__.lis

rm __dest__.lis

rm __diff__.lis

Supporting functions

The source directory, if a parameter was given, validate it is really a directory.

function validateSource

{

localSource=$1

if [ "$localSource" == "" ]

then

echo "Usage:"

echo " $0 <sourceDir> [<destinationDir>]"

echo " "

localSource=""

elif [ ! -d "$localSource" ]

then

echo "\"$localSource\" directory does not exist."

localSource=""

source=$localSource

}

The destination directory is not really required, but it is highly recommended. If it was not wiven, a default backup directory is selected.

function validateDest

{

localDest=$1

if [ "$localDest" == "" ]

then

localDest=/bak

if [ ! -d "$localDest" ]

then

mkdir "$localDest"

dest=$localDest

}

Sometimes we will need to copy files from source to destination. In case of directory, create it in destination, in case of file, copy and set the last changed date to the one of source, this can be done with TOUCH.

function copySource2Dest

{

# if it is a directory, create it

# if it is a file, copy it

if [ -d "$1/$3" ]

then

mkdir "$2/$3"

else

cp "$1/$3" "$2/$3"

if [ -f "$2/$3" ]

then

touch -r"$1/$3" "$2/$3"

}

When a file or directory was deleted from source, do the same in destination. Before deleting, validate the file exists in destination.

function delDest

{

if [ -d "$1/$2" ]

then

rm -R "$1/$2"

elif [ -f "$1/$2" ]

then

rm "$1/$2"

}

To compare source and destination files, we use the properties returned by the STAT command. The returned values will be used by the script to detect changes.

function getFileProps

{

export $(stat -s "$1/$2")

}

Installation

I have copied the script to /bin/usr in order to make the script available to be used within any directory. Otherwise, the script need to be in the same directory where it is called or within the PATH.

Future development

A way to maintain previous versions of files is highly recommended. Once versioning is implemented, also the restore mechanism will be required.

So far the script is working fine, still, any suggestion is well received.

The Script

Here you can find the script

Scripts - Simple scripts for Backup in Windows using MSDOS

2011-06-20T12:00:00.000-07:00

Download all the scripts

Scheduled tasks - Simple scripts for Backup in Windows using MSDOS

2011-06-19T12:00:00.000-07:00

To automatically create a daily incremental backup I’ve schedule a task in windows’ Scheduled Tasks. The only setting are the script to run and the recurrent schedule, by default, the script will do the rest. For my PC, I’m using a daily execution at 5PM.

The script will be:

D:\>Backup.cmd

Before the first execution, I performed a Full backup.

D:\>Backup.cmd full

FullRestore.cmd - Simple scripts for Backup in Windows using MSDOS

2011-06-18T12:00:00.000-07:00

The last option we need in order to have a [almost] complete backup tool is the full restore.

As part of the backup process there is a file with the list of all files per directory. We are going to use those files to restore all the newest files from the backup sets.

It might be the case that we need to restore the newest files but given an older list file (maybe because we delete some information we need to restore), in that case we set this variable.

set sourceFilesDir=<path where source LST files will be taken in a restore process>

if the variable is not set, the script looks for the newest backup set to be used as base for the LST files.

rem get newest lst files

if NOT exist "%sourceFilesDir%" (

FOR /F "usebackq delims==" %%i IN (`dir /oen/b/ad %backupDir%`) do (

set sourceFilesDir=%backupDir%\%%i

)

Of course, all this is after setting main variables.

Next thing is to retrieve all backup sets by date in descending order.

FOR /F "usebackq delims==" %%i IN (`dir /oe-n/b/ad %backupDir%`) do (

Most of the process is similar to previous restore scripts, with slight changes because we need to retrieve the file name in order to get it’s proper LST file and create the correct directory.

For example:

if /I "!fileName:~-10,5!"==".part" set process=0

...

if /I "!fileName:~-10,6!"==".part1" (

set process=1

set trimLen=10

)

We will later use those variables.

If the scripct receives a parameter, it means we need only to restore one directory, in that case, if the RAR file match the given parameter, we proceed with the process, otherwise we skip the file. The way to skip the file is using the process variable.

if NOT "%restoreOnly%"=="" (

if "!process!"=="1" (

set process=0

if "%restoreOnly%.rar"=="!fileName!" set process=1

if "%restoreOnly%.part1.rar"=="!fileName!" set process=1

if "%restoreOnly%.part01.rar"=="!fileName!" set process=1

if "%restoreOnly%.part001.rar"=="!fileName!" set process=1

if "%restoreOnly%.part0001.rar"=="!fileName!" set process=1

if "%restoreOnly%.part00001.rar"=="!fileName!" set process=1

)

If the file is suitable to be processed, we look for the proper LST file, then create an intermediary file, that’s because the original LST includes the Drive letter and we don’t need it to the restore.

rem decode original LST file

if /I "!trimLen!"=="4" set listFile=!fileName:~0,-4!.lst

if /I "!trimLen!"=="10" set listFile=!fileName:~0,-10!.lst

if /I "!trimLen!"=="11" set listFile=!fileName:~0,-11!.lst

if /I "!trimLen!"=="12" set listFile=!fileName:~0,-12!.lst

if /I "!trimLen!"=="13" set listFile=!fileName:~0,-13!.lst

if /I "!trimLen!"=="14" set listFile=!fileName:~0,-14!.lst

rem create the list of files to recover

if exist "%restoreDir%\RestoreFiles.lst" del "%restoreDir%\RestoreFiles.lst"

FOR /F "eol=; tokens=1 delims==" %%x in (%sourceFilesDir%\!listFile!) do (

set stringLine=%%x

@echo !stringLine:~3!>> "%restoreDir%\RestoreFiles.lst"

)

Now use the already set variables to extract all the required files.

"%rarPath%\rar.exe" x -o- -n@"%restoreDir%\RestoreFiles.lst" "%backupDir%\%%i\%%x" * %restoreDir%\ >> %restoreDir%\Restore.log

The used parameters are:

x = extract

-o- = do not overwrite

-n@<file> = extract only thise files listed in the given file

"%backupDir%\%%i\%%x" = RAR file

* = extract all (view -n@<file>)

%restoreDir%\ = targe dir

The usage is:

D:\>FullRestore.cmd

If we need only to recover one directory we use:

D:\>FullRestore.cmd temp

Where temp is the directory.

RestoreAll.cmd - Simple scripts for Backup in Windows using MSDOS

2011-06-17T12:00:00.000-07:00

Ok, we have just restore the newest version of the file we need form all backup sets, but it was not the one we were looking for. In this case we need to validate maybe the previous version.

This script restores all available files. Actually the script is the same as RestoreNewest, but with an extra directory in the RAR command.

if "!process!"=="1" (

"%rarPath%\rar.exe" x -o+ "%backupDir%\%%i\%%x" "%processfile%" %restoreDir%\%%i\ >> %restoreDir%\Restore.log

)

The usage is:

D:\>RestoreAll.cmd local*

Where local* is the file name we are looking for. The file can contain wildcards.

RestoreNewest.cmd - Simple scripts for Backup in Windows using MSDOS

2011-06-16T12:00:00.000-07:00

Similar to the functionality of ListBackup, but in this case we only restore the newest file.

Basically, it takes all backup sets, order the sets by date in a descendant manner, and then extract the required files from all available backups without overwriting. As we first process the newest backup set, we assure that the newest file is restored.

Most of the script is similar to ListBackup.cmd bun in this case we restore the contents mnot only list it.

For the restore, we ser a new variable:

set restoreDir=%restoreDrive%Restore_.%backupExt%

In this case the command is:

if "!process!"=="1" (

"%rarPath%\rar.exe" x -o- "%backupDir%\%%i\%%x" "%processfile%" %restoreDir%\ >> %restoreDir%\Restore.log

)

The new switch for WinRAR is

o- Do not overwrite existing files

The usage is:

D:\>RestoreNewest.cmd local*

Where local* is the file name we are looking for. The file can contain wildcards.

ListBackup.cmd - Simple scripts for Backup in Windows using MSDOS

2011-06-15T12:00:00.000-07:00

Once we have backup our files there will be the occasion we need to recover a file, but, what file to restore?

Using this script we can list the contents of all available backup (RAR) files, looking for a given file name.

Within the script, we first set local variables, after that, we find the fin the backup directory and every directory within; each inner directory represents a backup day. This is easy task because the backup directory has a fixed name and a DIR command gives us the internal directories

FOR /F "usebackq delims==" %%i IN (`dir /oe-n/b/ad %backupDir%`) do (

For each directory, we find the containing RAR files, again a DIR command helps us with it.

FOR /F "usebackq delims==" %%x IN (`dir "%backupDir%\%%i\*.rar" /oen/b`) do (

The format of RAR files, when they are split in several archives is:

<main RAR name>[.part<consecutive number>].rar

For example, for a single file: temp.rar

First file of a sequence: temp.part1.rar

In case we find “part” as part of the name, it is almost sure we need to skip that file, unless is the first one of the sequence. The first file of the sequence is named as:

<main RAR name>.part<several zeroes>1.rar

Looking for the word “part” in the file name will be:

if "!fileName:~-10,5!"==".part" set process=0

if "!fileName:~-11,5!"==".part" set process=0

if "!fileName:~-12,5!"==".part" set process=0

if "!fileName:~-13,5!"==".part" set process=0

if "!fileName:~-14,5!"==".part" set process=0

if "!fileName:~-10,6!"==".part1" set process=1

if "!fileName:~-11,7!"==".part01" set process=1

if "!fileName:~-12,8!"==".part001" set process=1

if "!fileName:~-13,9!"==".part0001" set process=1

if "!fileName:~-14,10!"==".part00001" set process=1

We set process to 0 when the file is a split file, but set it to 1 when is the first one.

If the file is suitable for process we execute the WinRAR command to list contents given a filter name.

if "!process!"=="1" (

"%rarPath%\rar.exe" vb "%backupDir%\%%i\%%x" "%processfile%"

)

The parameters user are:

C:\Program Files\WinRAR>rar

Shareware version Type RAR -? for help

Usage: rar <command> -<switch 1> -<switch N> <archive> <files...>

<@listfiles...> <path_to_extract\>

v[t,b] Verbosely list archive [technical,bare]

The usage is:

D:\>ListBackup.cmd local*

Where local* is the file name we are looking for. The file can contain wildcards.

Backup.cmd - Simple scripts for Backup in Windows using MSDOS

2011-06-14T12:00:00.000-07:00

For the easiest use and development, the backup should take all directories of the disk and then try to backup their contents. Depending of the backup type, it will backup all information or only those files updated since last backup.

In Windows is easy to know which files were updated, it just take us to review the file’s attributes and find if it has the Archive file attribute active. For example, the attributes for the file autoexec.bat

C:\>attrib autoexec.bat

A C:\autoexec.bat

Once we have backed up a file we will need to reset the Archive attribute, but, as I mention earlier, the WinRAR tool help us in the process. If we set the proper attributes to WinRAR, it will compress only files ready to archive and at the end it will also reset that attribute.

At the end, the backup script needs to set the proper parameters and then call WinRAR to do the job. Part of the required setting is to store each backup set in a separate directory to avoid overwriting the previous one.

Taking a look to the script we have that, after setting the variables, we check the parameters to determine whether it is a full o incremental type of backup.

if NOT "%1"=="" set full=1

Also, we need to create a proper backup dir according with the current date

set backupExt=bkp

set backupRoot=%backupDrive%Backup00.%backupExt%

set backupDir=%backupRoot%\%date:~6,4%%date:~3,2%%date:~0,2%.%backupExt%

if NOT exist "%backupRoot%" md "%backupRoot%"

if NOT exist "%backupDir%" md "%backupDir%"

backupDir will hold all backup files.

To do the process we need to get all directories, the dir command gives us that information. We just need to use it in a FOR loop

FOR /F "usebackq delims==" %%i IN (`dir /b/ad-h-s`) do (

But if we are processing the same drive where the backups are stored we need to avoid creating a backup of the backup files. This validation in an easy task because the main backup directory has a fixes name.

set localFile=%%i

set localextension=!localFile:~9!

if Not "!localextension!" == "%backupExt%" (

Remember the configuration file? Well, we need to validate if the current directory is not in the exceptions. This is made comparing the current directory against the values in the exceptions variable; again, this is made with a loop, extracting the comma separated values from the variable. As we need to have a limit in the search, we can evaluate as much as 100 exceptions.

rem validate is not an exception

set localExceptions=

set nextExceptions=%exceptions%

rem up to 100 exceptions

set maxExceptions=100

FOR /L %%e IN (0,1,!maxExceptions!) DO (

set localExceptions=!nextExceptions!

for /f "tokens=1,* delims=, " %%a in ("!localExceptions!") do (

set currentException=%%a

set nextExceptions=%%b

)

if /I "%%i"=="!currentException!" set processDirectory=0

if NOT "!nextExceptions!"=="" set maxExceptions=1

)

The script use the processDirectory variable as a switch to know if the directory will be processed or not.

Once we decide to process a directory, we create a list of files of it, this fill will be used later in a restore process.

dir /oen/b/s/a-d "%%i" > "%backupDir%\%%i.lst"

At the beginning of the script we set a variable to know if it was a full backup, now we will use it. An IF statement using the variable full help us to set the proper parameters for WinRAR.

if "%full%"=="0" (

"%rarPath%\rar.exe" a -ac -ao -r -m0 -v%maxRARSize%k "%backupDir%\%%i.rar" "%%i" > "%backupDir%\%%i.log"

) else (

"%rarPath%\rar.exe" a -ac -r -m0 -v%maxRARSize%k "%backupDir%\%%i.rar" "%%i" > "%backupDir%\%%i.log"

)

Basically the main logic is to select which directories to backup and to set the backup destination directory. Al the other job is made by WinRAR.

The WinRAR parameters used are:

C:\Program Files\WinRAR>rar

Shareware version Type RAR -? for help

Usage: rar <command> -<switch 1> -<switch N> <archive> <files...>

<@listfiles...> <path_to_extract\>

a Add files to archive

ac Clear Archive attribute after compression or extraction

ao Add files with Archive attribute set

r Recurse subdirectories

m<0..5> Set compression level (0-store...3-default...5-maximal)

v[k,b] Create volumes with size=*1000 [*1024, *1]

The compression used os only to store the files. This makes our backup faster but it use more space. Changing that switch will save space but with slower backups.

The way to use the script will be as follows:

For a full backup:

D:\>Backup.cmd full

For a incremental backup:

D:\>Backup.cmd

With the default configuration, you will get the next directory(for the date of Jun 6th, 2011):

D:\Backup00.bkp\20110616.bkp

Within that directory you will find LST files containing the list of all files per directory, LOG files containing the operation results per directory and the RAR files containing the backup files.

BackupConfig.cmd - Simple scripts for Backup in Windows using MSDOS

2011-06-13T12:00:00.000-07:00

This script prepares the required variables for the backup operation, this file is optional for the process, if it does not exist, the process will use default values. But if it does exist, it needs to be correctly configured.

The configuration file is only called by other scripts; it has no use to be called directly

There are few variables to set:

Path where RAR.exe is stored

set rarPath=C:\Program Files\WinRAR

Drive to backup

set backupDrive=D:\

Drive to sotre restored files

set restoreDrive=D:\

Path where source LST files will be taken in a restore process

set sourceFilesDir=

Max size for rar files, specify KB

set maxRARSize=1000000

Comma separated list of directories to skip, up to 100 exceptions

set exceptions=,Temp,Oracle,app,

Simple scripts for Backup in Windows using MSDOS

2011-06-11T12:00:00.000-07:00

Not long ago I was reviewing [and cleaning] my files from my PC, in the process I review the files and move them to another location supposedly easiest to remember and access. About a month after the process and after updating all my backups I realize I lost about 1GB of information form an old project. It might not been a big issue because it was more than 2 years old information but it was a lost after all.

A couple of months later I was working in a presentation for a project that has daily changes. One day I tried to go back a couple of versions but I was unable to get the one I want. At that time, and still now, my versioning was a number at the end of the file’s name, but it is not a reliable process.

I came to the conclusion that I needed a sort of daily incremental backup process, to keep track of my files.

I started reviewing backup software but besides the cost, most of them requires an external device, installation and are like black boxes for end users.

As I like challenges, I start my own backup project with these premises.

It needs to:

Backup all files (full backup)
Backup only changed files (incremental backup)
Restore the newest file from backup (given the name)
Restore any version of a file from backup history (given the name)
Restore all files from backup (the newest of each file)
Use local disks or external devices (attaches to the PC and as named disks)
Compress the backup files to save space
Only backup data files

My PC is configured with 2 partitions, C for Windows OS and D for all my data. The intention for the backup is to cover only my data files. The final product might be used to backup system files also but those won’t work for a full OS recovering.

Starting with the compression requirement, I made tests for compression using WinRAR, and thanks to RARLAB for this great software. Part of the functionality of WinRAR is a king ob Backup system.

Exploiting WinRAR Functionality and its command line version, I made the next scripts to accomplish my requirements...

The final solution includes these files:

BackupConfig.cmd
Backup.cmd
ListBackup.cmd
RestoreNewest.cmd
RestoreAll.cmd
FullRestore.cmd

Next, we will talk about the functionality.

2011-03-23T10:28:00.000-07:00

I’ve just saw an interesting interview that HBR Video Idea Cast made to Marshall Goldsmith, in case you haven’t heard about him, he is the author of the book What Got You Here Won't Get You There (here the link to Amazon) and a blogger within HBR site

During the interview, Mr. Goldsmith speaks about leaders’ characteristics which have remain during time and other which have change and might disappear in the future. Also, he talks about some strategies to influence our bosses and peers. Despite the fact the interview is kind old, from June 2008, the spoken topics are important and currently relevant. I think this is a great interview which need to be reviewed by those who currently are or want to be better leaders.

Here you can have the video, the file is an MP4 of 92MB length.

There are other interesting interviews made by HBR, you can find the RSS feed for the Video Idea Cast here. You can also subscribe to the feed using an iTunes University link

Is it really possible to multi-task?

2011-02-15T20:00:00.000-08:00

I’m subscribe to a Discovery Channel Podcast named “FRIDAY NEWS FEEDBAG”, in a recent program they talked about multitasking in humans , the talk included an interview to a scientist explaining the topic. Basically we cannot multitask activities that use the same part of the brain, mainly we are switching activities.

As far as I can explain, we can only multitask things that we do almost automatically, let’s say walk and breathe at the same time.

You can find the post of the podcast here.

If you are interested in he audio, here is it.

Here the RSS feed.

HBR IdeaCast

2011-01-04T18:06:00.000-08:00

Recently, I started listening to the HBR IdeaCast’s Podcast. It is really a good source of references and ideas. On its format, they choose a topic and then developed using interviews and author insights. As far as all the episodes I have already listen I’m too satisfied with the content.

In case you want to review the audios, this is the link: http://blogs.hbr.org/ideacast/

Avoid uploading duplicated files in FTP using MSDOS script

2010-11-18T12:00:00.000-08:00

I was uploading files using the script I made in a previous post, and all was working smoothly until I reach a directory full of images used for an address book application, there were about 2,000 files. The current version of the script made no comparison for file existence (actually it deletes the old file in the server before attempting to save the new one) or date comparison. In this case, and assuming those files change don’t change too often, I start thinking in a new process to remove the duplicated files and only upload the non existing files.

I made a new FTP script to read the remote files and then delete the existing files from the local PC. This process is a dumb process and only task is to delete all files from the list. It is not aware of date changes or new file versions. Saying these, if you use this script, always work with a backup of your files, never with your original files as they might be deleted.

The script usage is:

deleteDuplicatedFiles.cmd <Root directory to validate>

Current version of the script has no recursive processes implemented

The operation is as follows:

1. Set working variables and defaults

set user=<ftp user>

set pass=<ftp password>

set site=<ftp site>

set base=<remote path to validate>

set file=<FTP commands file>

set outputFile=<local file to store file listing>

2. Start creating the commands file

echo user %user% %pass% >> %file%

echo cd %base% >> %file%

echo ls >> %file%

echo bye >> %file%

3. Execute the commands´ file

ftp -n %site% < %file% > %outputFile%

4. Call to delete the preexisting files

call :deleteFiles "%workDir%" %outputFile%

5. The deleteFiles subroutine reads all lines from the output file and, assuming is a file name, it validate its existing using another subroutine

FOR /F "usebackq delims==" %%i IN (`type %localFile%`) do (

call :deleteSingleFile %%i

)

6. I’m using deleteSingleFile in case another validation is required or needed to implement, currently it only check the file existence and then delete the file

if exist %deleteFile% del %deleteFile%

Here you can find a copy of the script in case you want to try it: deleteDuplicatedFiles.zip

DISCLAIMER: this script is delivered as is and I take no responsibility of any deleted or damage file. Use this script cautiously and always work on a backup to preserve the original files.

FTP Uploading files using MSDOS script

2010-11-11T12:00:00.000-08:00

Recently I was working in a static Web Site and at the time I need to upload it to the server, I realize there were several files and directories to upload. As I’ve never been a big fan of FTP programs other than the command line provided in each OS, and, instead of searching if there were already a public solution, I started thinking in an automation script to upload the files.

The basic idea of an automation script is to read the contents of a directory, create a commands script for FTP and then execute the generated script. But it also needs to be recursive within internal directories.

As I was working with Windows, the script I created is for MS DOS

The way to call it is:

uploadfiles <Root directory to upload>

Where the given root directory is where the desired files to upload are stored.

Taking a little walkthrough of the script, it works like these:

1. Set working variables and defaults.

set user=<set FTP User>

set pass=<FTP Password>

set site=<site to connect>

set base=<target directory>

set file=<temporary commands file>

2. Next thing to do is to start creating the commands file, some of the used values have been set in step 1

echo user %user% %pass% >> %file%

echo cd %base% >> %file%

echo pwd >> %file%

echo bin >> %file%

echo hash >> %file%

3. The directory processing logic was implemented in a subroutine, we use it to process the root directory

call :processDir . "%workDir%" 0 %file% x

4. Once the directory was processed, we return just to call the FTP to use the commands file and upload all found files.

ftp -n %site% < %file%

The processDir subroutine is called recursively to process all found subdirectories. Once the script is finished, it is intended to work within a FTP session, once in the session if might change directories in remote site, but not in local PC, so the script need to handle all calls to local files as a relative path from where the script was called. This routine works as follows:

1. Local variables are set, from received parameters

set localPath=%~1

set localDir=%~2

set localID=%3

set localFile=%4

set quotes=1

set fullpath=%localPath%\%localDir%

2. For each processed directory, the script assumes that all need to be created

echo mkdir %localDir% >> %localFile%

echo cd %localDir% >> %localFile%

echo pwd >> %localFile%

3. The is finds all the directories of the current one and call itself to process them

FOR /F "usebackq delims==" %%i IN (`dir /b/ad`) do (

...

if "!localquote!"=="0" call :processDir %fullpath% %%i 1 %localFile% %5x

if "!localquote!"=="1" call :processDir "%fullpath%" "%%i" 1 %localFile% %5x

)

4. Next step is to process all files form current directory

FOR /F "usebackq delims==" %%i IN (`dir /b/a-d`) do (

...

if "!localquote!"=="0" (

echo ls %%i >> %localFile%

echo dele %%i >> %localFile%

echo put %fullpath%\%%i >> %localFile%

)

if "!localquote!"=="1" (

echo ls "%%i" >> %localFile%

echo dele "%%i" >> %localFile%

echo put "%fullpath%\%%i" >> %localFile%

)

The process is always trying to detect is the files are received with quotes or spaces within its name to make the proper handle, but, I haven’t fully test it for such cases.

Testing the script, let’s say we have the script and the desired directory to upload (web) in d:\temp, we execute it like this:

D:\Temp>uploadFiles.cmd Web

The screen output, in this case, is:

Processing .\Web\Administration

Processing .\Web\bin

Processing .\Web\Controls

Processing .\Web\HtmlTemp

Processing .\Web\Images

Processing .\Web\JS

Processing .\Web

Processing FTP

Having configured the script with these values:

set user=myUser

set pass=myPassword

set site=ftp.mySite.com

set base=/www/

set file=\command.ftp

The created commands file is:

user myUser myPassword

cd /www/

pwd

bin

hash

mkdir Administration

cd Administration

pwd

ls CountryAccess.aspx

dele CountryAccess.aspx

put .\Web\Administration\CountryAccess.aspx

ls CountryAccess.aspx.cs

dele CountryAccess.aspx.cs

put .\Web\Administration\CountryAccess.aspx.cs

ls CountryAccess.aspx.designer.cs

dele CountryAccess.aspx.designer.cs

put .\Web\Administration\CountryAccess.aspx.designer.cs

...

pwd

mkdir JS

cd JS

pwd

ls flash_detect.js

dele flash_detect.js

put .\Web\JS\flash_detect.js

ls GeneralFunctions.js

dele GeneralFunctions.js

put .\Web\JS\GeneralFunctions.js

cd ..

pwd

ls Web.config

dele Web.config

put .\Web\Web.config

cd ..

pwd

Here you can find a copy of the script in case you want to test it yourself: uploadFiles.zip

Daily accomplishments

2010-08-27T12:00:00.000-07:00

I was reading “No B.S. Ruthless Management of People and Profits: The Ultimate, No Holds Barred, Kick Butt, Take No Prisoners Guide to Really Getting Rich” by Dan Kennedy.

It makes reference to a study made to top executives.

According with the referenced study, the executives, by their own account only averages 40 minutes or less of actual accomplishment.

In my work as a Project Manager, we found an average developer to be productive for 80% of their time, in an 8 hours shift, it will be about 6:30 hours; plans where made with this in mind.

Could that be that when responsibilities are increased and more people need to be managed by us, our own accomplishments decrease?

iPhone's Numeric Pad keyboard with a DOT

2010-08-13T12:00:00.000-07:00

I was developing an application for iPhone which handl

es numeric fields to register information. To my disappointment, even in iOS 4, there is no default keyboard for a Number pad including a dot “.”; there is a Number pad with numbers from 0 to 9 and the backspace, in the other hand we have the Numbers and punctuations, which includes numbers, dot, parenthesis, and other symbols. T

he problem with the former is the absence of a dot and the later is that we can easily switch to the alphabetical keyboard.

But what happen with the Number pad including a dot? The Number pad keyboard includes a blank space

After a little search a found a couple of examples of including the dot in the kwyboard, using the blank space.

First, lets say we have a text field defined and connected to the interface

IBOutlet UITextField * textMoney;

And wich keyboard type is the Number Pad

self.textMoney.keyboardType = UIKeyboardTypeNumberPad;

In order to include the Dot, we need to stat receiving notifications to act propperly when the keyboard shows:

/**

View will appear

**/

- (void)viewWillAppear:(BOOL)animtated

{

if ([[[UIDevice currentDevice] systemVersion] floatValue] >= 3.2)

{

[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(keyboardWillShow:) name:UIKeyboardDidShowNotification object:nil];

}

else

{

[[NSNotificationCenter defaultCenter] addObserver:self selector:@selector(keyboardWillShow:) name:UIKeyboardWillShowNotification object:nil];

}

We include the notification, according with the detected iOS.

/**

View will disappear

**/

- (void)viewWillDisappear:(BOOL)animtated

{

[[NSNotificationCenter defaultCenter] removeObserver:self];

}

The dismiss te notification once the view is not showed.

Now, all interesting process begin with when we received the notification in the configured function, in this case keyboardWillShow

/**

The Keyboard will be shown

**/

- (void)keyboardWillShow:(NSNotification *)notification

{

if ([textNotes isFirstResponder])

{

// Skip for Notes keyboard

return;

}

// Locate fist view

UIWindow *tempWindow = [[[UIApplication sharedApplication] windows] objectAtIndex:1];

UIView *keyboard;

// Define Button

UIButton * utilityButton = [UIButton buttonWithType:UIButtonTypeCustom];

utilityButton.frame = CGRectMake(0, 163, 105, 53);

// Set Text and font

[utilityButton.titleLabel setFont:[UIFont systemFontOfSize:35]];

[utilityButton setTitle:[Dot or Comma according with localized settings] forState:UIControlStateNormal];

// Set Solors

[utilityButton setTitleColor:[UIColor colorWithRed:77.0f/255.0f green:84.0f/255.0f blue:98.0f/255.0f alpha:1.0] forState:UIControlStateNormal];

[utilityButton setTitleColor:[UIColor whiteColor] forState:UIControlStateHighlighted];

// Background for the pressed state

[utilityButton setBackgroundImage:[UIImage imageNamed:@"background.png"] forState:UIControlStateHighlighted];

// Add call to functionality

[utilityButton addTarget:self action:@selector(addDecimalPointToField) forControlEvents:UIControlEventTouchUpInside];

// Behavior when orientation is changed

[utilityButton setAutoresizingMask:(UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleRightMargin | UIViewAutoresizingFlexibleTopMargin | UIViewAutoresizingFlexibleHeight)];

// Locate keyboard view and add button

for(int i=0; i<[tempWindow.subviews count]; i++)

{

keyboard = [tempWindow.subviews objectAtIndex:i];

if ([[[UIDevice currentDevice] systemVersion] floatValue] >= 3.2)

{

if([[keyboard description] hasPrefix:@"<UIPeripheralHost"] == YES)

{

[keyboard addSubview:utilityButton];

}

else

{

if([[keyboard description] hasPrefix:@"<UIKeyboard"] == YES)

{

[keyboard addSubview:utilityButton];

}

Again, we need to make a distinction for the iOS.

One little extra t

hing, to add the dot to the current field with the focus, we use a trick, copy the dot to memory and then call the paste function of the current field, this is done in addDecimalPointToField.

/**

Send Decimal Point event to field with focus

**/

- (void)addDecimalPointToField

{

UIView * objectToEdit = nil;

// Locate field with the focus

for (UIView * localView in self.scrollView.subviews)

{

if ([localView isFirstResponder])

{

objectToEdit = localView;

}

// Try to insert dot

if (objectToEdit != nil)

{

// Detect if the Dot has been already inserted

NSString * localText = [(UITextField *)objectToEdit text];

NSRange separatorPosition = [localText rangeOfString:[Dot or Comma according with localized settings]];

if (separatorPosition.location == NSNotFound)

{

// Dot wasn't found, add it

[objectToEdit insertText:[NSString stringWithFormat:[Dot or Comma according with localized settings]]];

}

I have change part of the code with [According with the localization options, add a Dot or a Comma], this in order to let you decide what punctuation symbol to use.

The result will be something like:

I'm not really sure it this will be accepted by Apple to be release to iTunes Store, I haven't use it for a commercial application, at least, my AdHoc application implements it.

Hope this help you.

Migrating Contacts over phones

2010-08-02T12:00:00.000-07:00

Recently I was helping in a cell phone migration; contacts were the most important thing to migrate, but the devices where not directly compatible as the devices where a Nokia and a Samsung.

After a couple of Google’s searches, I got to the idea that there were no application so make my life easier, so I got to the task to investigate what the vendor’s applications were capable of.

I install in a Windows PC both suites, Nokia PC Suite and Samsung PC Studio, after I connect and synchronize both devices I got the option of exporting and importing contacts.

From the Nokia PC Suite, I exported all contacts; as a result I got a single file including all the 100+ contacts. I then turn to Samsung’s suite and try to import from the file, but it only imports the first one.

I don’t know if it has something to do with the vCard standard, but, most of the products I´ve tried do the same when reading multiple cards within a file.

So, next thing to do was to create as many files as contacts to import, wait a minute, this is more than 100 files!

What if I want to make a migration of a device with thousand of contacts?

To make my life easier, I start creating a DOS shell script to separate the file in as many contacts as is has. Its operation will be easy:

1. Read source file line by line

2. When detects the tag “BEGIN:VCARD” create a new file

3. Writes all read lines to the created file

Whit this simple process I got all the files I need and next step was to imported in Samsung’s suite. Fortunately, it support to drag-n-drop many files at a time, so, just in a moment the Agenda was introduced in the new phone.

Here is the code to create the vCards.

It reads the input file in the same path. File with contacts need to be named as “contacts.txt”

@echo off

set fileNumber=0

setLocal EnableDelayedExpansion

for /f "tokens=* delims==" %%a in ('type contacts.txt') do (

if "BEGIN:VCARD" == "%%a" (

set /a fileNumber+=1

echo BEGIN:VCARD > !fileNumber!.vcf

)

if NOT "BEGIN:VCARD" == "%%a" (

echo %%a >> !fileNumber!.vcf

)

goto :eof

Here you can find a copy of the script in case you want to test it yourself: readCards.zip

WebServices solution for iPhone - Second Stage

2010-07-08T12:00:00.000-07:00

The FileDownloadController class

Once I had the FileDownload class, it came to me the problem of using it many times and from many places. I could create as many FileDownload objects as needed, but this will create many variables and lines of code. My solution was to create a controller class (FileDownloadController), which I implement with a Singleton Pattern.

The concept is:

The FileDownloadController class will create and manage all needed FileDownload objects
Whenever an object needs to download a file, it should get the FileDownloadController instance and then asks for the download
Once the download is complete, or in case of error, a callback will be received by the requesting object

In order to maintain certain acceptable performance, because the download process works in the background and consumes resources, there should be a limited number of concurrent downloads, this number is set when the controller is created. In my tests, using an iPhone 3G, an acceptable number is 10.

Because of the limited number of download slots, we might face the problem of having more requests than slots, for this we have 2 possible solutions:

Store requests in a queue and attend them once a slot is available
Cancel the oldest download and use it for the requested

As one of the purposes of these classes were to download images, and those been requested from a scrollable table, the selected solution was number 2.

To attend this new requirement, an intermediary class was created to hold the FileDownload object and the time when the process was called. The interface of this class is:

@interface FileDownloadControllerItem : NSObject

{

int jobID;

FileDownload * downloadObject;

NSDate * startTime;

}

@property (nonatomic, retain) FileDownload * downloadObject;

@property (nonatomic, assign) int jobID;

@property (nonatomic, retain) NSDate * startTime;

-(FileDownloadControllerItem *)initwithDelegate:(id)delegate;

-(void)downloadFile:(NSInteger)fileID fileName:(NSString *)name callBack:(id<FileDownloadProtocol>)delegate;

-(void)stopDownload;

@end

jobID it holds the Download ID discussed in the previous post of this series.
downloadObject holds the FileDownload object that will be used to process the download
startTime holds the time when the download was called and will be used to identify the older download when a new download is requested and there are no available slots

Its implementation is pretty straightforward and in most cases are direct calls to the FileDownload object.

-(FileDownloadControllerItem *)initwithDelegate:(id)delegate;

Create objects and set the delegate for callbacks

-(void)downloadFile:(NSInteger)fileID fileName:(NSString *)name callBack:(id<FileDownloadProtocol>)delegate;

Is called when a file needs to be download, it set the self variables and call the FileDownload object to start the process. This class is not validating any available slot; it assumes that validation was made previously.

-(void)stopDownload;

Just call to cancel the current download.

About the implementation

/**

Initializtion given a delegate

**/

-(FileDownloadControllerItem *)initwithDelegate:(id)delegate

{

[self init];

self.jobID = FREESLOT;

self.downloadObject = [[FileDownload alloc] init];

self.startTime = [NSDate dateWithTimeIntervalSince1970:0];

return self;

}

/**

Download a file

**/

-(void)downloadFile:(NSInteger)fileID fileName:(NSString *)name callBack:(id)delegate;

{

self.jobID = fileID;

self.startTime = [NSDate dateWithTimeIntervalSinceNow:0];

[self.downloadObject setJobID:fileID];

[self.downloadObject startDownloadFile:name callBack:delegate];

}

/**

Cancel current download

**/

-(void)stopDownload

{

[self.downloadObject stopDownload];

}

Most of the logic is implemented in the Controller class, which interface is as follows:

@interface FileDownloadController : NSObject

{

int currentDownloads;

BOOL maintainQueue;

int totalJobs;

NSMutableArray * downloadArray;

NSMutableDictionary * callBackObjects;

}

@property (nonatomic, assign) int totalJobs;

@property (nonatomic, assign) BOOL maintainQueue;

+ (FileDownloadController *)getController;

+ (FileDownloadController *)getController:(NSInteger)jobs useQueue:(BOOL)queue;

- (FileDownloadController *)initWithJobs:(NSInteger)jobs;

- (FileDownloadController *)initWithJobs:(NSInteger)jobs useQueue:(BOOL)queue;

- (BOOL)downloadFile:(NSInteger)fileID fileName:(NSString *)name callBack:(id<FileDownloadProtocol>)delegate;

- (BOOL)findJobInQueue:(NSInteger)job;

- (void)cancelDownloadFile:(NSInteger)fileID;

- (void)stopAllDownloads;

@end

currentDownloads is used to keep track of the actual concurrent downloads
maintainQueue is not used, yet, but it intention is to identify if a download queue is used or if the older download might be cancelled
totalJobs maximum number of concurrent downloads
downloadArray this array holds all the download slots
callBackObjects is used to hold the download ids and the callback objects; this is needed because the FileDownload will call a callback within the controller, then the controller will call the requesting object

The available functions are:

+ (FileDownloadController *)getController;

This is used to retrieve a controller with default values, internally it calls initWithJobs with 10 jogs and not using the queue

+ (FileDownloadController *)getController:(NSInteger)jobs useQueue:(BOOL)queue;

Is used to create a controller with specific values, it also calls initWithJobs with the given parameters

- (FileDownloadController *)initWithJobs:(NSInteger)jobs;

Simple constructor which initialize the controller with the given jobs and not using the queue

- (FileDownloadController *)initWithJobs:(NSInteger)jobs useQueue:(BOOL)queue;

Initialize the controller with the given jobs and queue use, this function creates the required download slots

- (BOOL)downloadFile:(NSInteger)fileID fileName:(NSString *)name callBack:(id<FileDownloadProtocol>)delegate;

Called to download a file

- (BOOL)findJobInQueue:(NSInteger)job;

As each download required an ID, this function tells us if an ID is already in the download queue, this is, if it was found in any download slot

- (void)cancelDownloadFile:(NSInteger)fileID;

Call to cancel a given download

- (void)stopAllDownloads;

Called to stop all downloads, mainly called from the destructor

There are other internal functions used by the class, the implementation is:

/**

Get Singleton object

**/

+ (FileDownloadController *)getController

{

if (controller == nil)

{

controller = [[FileDownloadController alloc] initWithJobs:10 useQueue:NO];

}

return controller;

}

/**

Get Singleton object

**/

+ (FileDownloadController *)getController:(NSInteger)jobs useQueue:(BOOL)queue

{

if (controller == nil)

{

controller = [[FileDownloadController alloc] initWithJobs:jobs useQueue:queue];

}

return controller;

}

Both functions call the constructors.

/**

Initialization with job number

**/

-(FileDownloadController *)initWithJobs:(NSInteger)jobs

{

return [self initWithJobs:jobs useQueue:NO];

}

/**

Initialization with jobs and Queue handling

**/

-(FileDownloadController *)initWithJobs:(NSInteger)jobs useQueue:(BOOL)queue

{

[self init];

totalJobs = jobs;

maintainQueue = queue;

currentDownloads = 0;

downloadArray = [[NSMutableArray alloc] initWithCapacity:self.totalJobs];

for (int i = 0; i < self.totalJobs; ++i)

{

[downloadArray addObject:[[FileDownloadControllerItem alloc] initwithDelegate:self]];

}

callBackObjects = [[NSMutableDictionary alloc] init];

return self;

}

Within the constructor, the download slots are created and object’s variables initialized.

The downloadArray holds the FileDownloadControllerItem created for each slot.

The callBackObjects will hold the relation of each download ID and the object related for the callback

/**

Download files

**/

-(BOOL)downloadFile:(NSInteger)fileID fileName:(NSString *)name callBack:(id<FileDownloadProtocol>)delegate

{

// Validate file is not already in download queue, cero is not valid id

// Find a free download slot

// If none, clear one

// Configure slot to download file

BOOL error;

FileDownloadControllerItem * downloadItem;

error = [self findJobInQueue:fileID];

if (!error)

{

[self storeSession:fileID callBack:delegate];

downloadItem = [self getFreeSlot:NO];

if (downloadItem == nil)

{

downloadItem = [self getFreeSlot:YES];

}

[self increaseDownloadCounter];

[downloadItem downloadFile:fileID fileName:name callBack:self];

}

return downloadItem != nil;

}

To start a download process, first thing to do is validate it is not already been downloaded. Then we store the session in the dictionary and look for a free slot ([[self getFreeSlot:NO]), if there is none, we get one canceling the oldest ([[self getFreeSlot:YES]). Finally we increase the download counter and start the download ([downloadItem downloadFile:fileID fileName:name callBack:self]).

/**

Find if job is already in progress

**/

-(BOOL)findJobInQueue:(NSInteger)job

{

BOOL error = NO;

FileDownloadControllerItem * downloadItem;

for (int i = 0; i < [downloadArray count]; ++i)

{

downloadItem = (FileDownloadControllerItem *)[downloadArray objectAtIndex:i];

if (downloadItem.jobID == job)

{

error = YES;

}

return error;

}

To find if a job is already been downloaded, we loop through all the slots and compare the IDs

/**

Cancel docwnload given its fileID

**/

-(void)cancelDownloadFile:(NSInteger)fileID

{

[self cancelDownload:[self getSlot:fileID]];

}

To cancel a download, just get the slot a a given ID and call to cancel it using cancelDownload

/**

Call to cancel download activities

**/

-(void)cancelDownload:(FileDownloadControllerItem *)downloadItem

{

if (downloadItem != nil)

{

[self deleteSession:downloadItem.jobID];

[self decreaseDownloadCounter];

[downloadItem.downloadObject stopDownload];

downloadItem.jobID = FREESLOT;

}

cancelDownload cancel the downloadItem, decrease counters and clear sessions for the canceled download.

/**

Stop downloads

**/

-(void)stopAllDownloads

{

for (int i = 0; i < [downloadArray count]; ++i)

{

FileDownloadControllerItem * downloadItem = (FileDownloadControllerItem *)[downloadArray objectAtIndex:i];

[self deleteSession:downloadItem.jobID];

[self decreaseDownloadCounter];

[downloadItem stopDownload];

downloadItem.jobID = FREESLOT;

}

To stop all downloads, loop the slots and call each to be cancelled.

/**

Get a worker given a jobid

**/

-(FileDownloadControllerItem *)getSlot:(NSInteger)job

{

FileDownloadControllerItem * downloadItem = nil;

for (int i = 0; i < [downloadArray count]; ++i)

{

downloadItem = (FileDownloadControllerItem *)[downloadArray objectAtIndex:i];

if (downloadItem.jobID == job)

{

return downloadItem;

}

return downloadItem;

}

This function finds a downloadItem given the ID and returns it

/**

Get a free worker

**/

-(FileDownloadControllerItem *)getFreeSlot:(BOOL)force

{

FileDownloadControllerItem * downloadItem = nil;

FileDownloadControllerItem * downloadItemReturn = nil;

NSDate * minDate = [NSDate dateWithTimeIntervalSinceNow:0];

for (int i = 0; i < [downloadArray count]; ++i)

{

downloadItem = (FileDownloadControllerItem *)[downloadArray objectAtIndex:i];

if ( ( downloadItemReturn == nil && downloadItem.jobID == FREESLOT ) || force)

{

if (force)

{

if ([downloadItem.startTime timeIntervalSince1970] < [minDate timeIntervalSince1970])

{

minDate = [NSDate dateWithTimeIntervalSince1970:[downloadItem.startTime timeIntervalSince1970]];

downloadItemReturn = downloadItem;

}

else

{

downloadItemReturn = downloadItem;

}

if (downloadItem != nil)

{

[self cancelDownload:downloadItemReturn];

}

return downloadItemReturn;

}

getFreeSlot is used to retrieve an available downloadItem, in normal execution, it loop through all the slots looking for an empty one. When forcing the availability, it finds the oldest, which will be cancelled and then returned as a free slot.

The controller implements the FileDownloadProtocol and need to implement its functionality, this is:

- (void) downloadError:(NSError *)error downloadID:(NSInteger)jobID;

- (void) downloadSucceed:(NSMutableData *)data downloadID:(NSInteger)jobID;

In both cases, it deletes the session created for the download and return the information to the object which request the download using a callback.

With all this work done, when an object requires to download a file, it need to do some like these:

downloadController = [FileDownloadController getController];

[downloadController downloadFile:fileID fileName:url callBack:self];

Implementing the FileDownloadProtocol we should have:

#pragma mark -

#pragma mark FileDownloadProtocol

/**

Notify an error

**/

- (void) downloadError:(NSError *)error downloadID:(NSInteger)jobID

{

// Notify error to screen

}

/**

Notify download complete

**/

- (void)downloadSucceed:(NSMutableData *)data downloadID:(NSInteger)jobID

{

// Do something with the data, remember the jobID is the ID sent in downloadFile

}

Using this class, a file download is easiest than previous. Now just remember to release the downloadController object before exiting the application.

in a next post I'll complete this series with the Third Stage: calling WebServices.

I used to read my eMail on my iPhone, but not any more...

2010-07-01T14:49:00.000-07:00

When I got my iPhone I configure it to sync all my email account, at first, maybe because on the novelty of the device, I used the vibrate notification, but is was too annoying. When I deactivate the alarm, I start checking it too often.

After a couple of weeks I definitely delete all mail account from the iPhone. I got to the next conclusions:

No email is too important to be notified immediately, if those cases they can still contact me by phone
Checking mail took to much time, I can use that time in other activities
I rather prefer to read emails in my PC Screen

Recently I found this post from HBR: Why I Returned My iPad, is kind the same that happens to me.

WebServices solution for iPhone - First Stage

2010-06-18T12:00:00.000-07:00

The FileDownload Class

After reviewing some examples of how to download a file from an URL and after implementing it by myself, I came to the idea to do it as a separate class. The objective of the new class was to handle the connection and processing needed to do the job.

The published methods are:

- (FileDownload *) initWithURL:(NSString *)url callBack:(id<FileDownloadProtocol>)delegate;

This initializer is used to immediately start the download process given the desired URL to download and the delegate, which is the object which will receive the downloaded data. Its implementation is as follows:

/**

Initialize with file to download

**/

- (FileDownload *)initWithURL:(NSString *)url callBack:(id<FileDownloadProtocol>)delegate

{

[self init];

[self startDownloadFile:url callBack:delegate];

return self;

}

I’m not checking if the object was created successfully before starting the download, that’s a to do activity for one day.

- (void)startDownloadFile:(NSString *)url callBack:(id<FileDownloadProtocol>)delegate;

This method is used to start the download process given the desired URL to download and the delegate, which is the object which will receive the downloaded data. Internally, this method is called from the initializer, when the initializer is used. Its implementation is as follows:

/**

Call to download file

**/

- (void)startDownloadFile:(NSString *)url callBack:(id<FileDownloadProtocol>)delegate

{

processing = YES;

callBackObject = delegate;

NSURLRequest *xmlURLRequest = [NSURLRequest requestWithURL:[NSURL URLWithString:url]];

self.dataFeedConnection = [[[NSURLConnection alloc] initWithRequest:xmlURLRequest delegate:self] autorelease];

if (self.dataFeedConnection == nil)

{

NSDictionary *userInfo = [NSDictionary dictionaryWithObject:NSLocalizedString(@"No Connection Error", @"Error message displayed when not connected to the Internet.") forKey:NSLocalizedDescriptionKey];

NSError *noConnectionError = [NSError errorWithDomain:NSCocoaErrorDomain code:kCFURLErrorNotConnectedToInternet userInfo:userInfo];

[self handleError:noConnectionError];

}

[UIApplication sharedApplication].networkActivityIndicatorVisible = YES;

}

Basically it is the well know recipe to download a file.

-(void)stopDownload;

This method is used to cancel any active download. If there is no active download, nothing is done. Its implementation is as follows:

/**

Stop actual download process

**/

-(void)stopDownload

{

processing = NO;

if (dataFeedConnection != nil)

{

[dataFeedConnection cancel];

}

I’m using the processing variable to know if something is happening and avoid calling downloadError method once the download was cancelled.

One important property of the class is:

@property (nonatomic, assign) int jobID;

This property is a logical way to identify the current download, used for the case when more than 1 instance of the class is used a same time.

The class implements the next protocol

@protocol FileDownloadProtocol

/**

Notify an error

**/

- (void) downloadError:(NSError *)error downloadID:(NSInteger)jobID;

/**

Notify download complete

**/

- (void) downloadSucceed:(NSMutableData *)data downloadID:(NSInteger)jobID;

@end

Both methods defined in the protocol are required. The protocol is used to define the delegate object that will handle the received data.

The protocol’s methods are called once the process is finished or an error happens. There are 2 options:

- (void) downloadError:(NSError *)error downloadID:(NSInteger)jobID;

This method is called back to the delegate when an error occurs. Currently, there is no way to propagate the kind of error raised, but for the class implementation I only need to know if the download was successfully or not. This is only called from the error handler method:

- (void)handleError:(NSError *)error

{

if (processing && callBackObject != nil)

{

// Call the error Callback

[callBackObject downloadError:error downloadID:jobID];

}

Thanks to processing, it is only called when a processing is active.

- (void) downloadSucceed:(NSMutableData *)data downloadID:(NSInteger)jobID;

This method is called when the download succeed; with this call, the class is passing the downloaded information to the delegate. This is called from the NSURLConnection delegate method connectionDidFinishLoading:

/**

Connection finishes

**/

- (void)connectionDidFinishLoading:(NSURLConnection *)connection

{

self.dataFeedConnection = nil;

[UIApplication sharedApplication].networkActivityIndicatorVisible = NO;

if (processing && callBackObject != nil)

{

// Call the Succeed Callback

[callBackObject downloadSucceed:dataData downloadID:jobID];

}

self.dataData = nil;

}

Internally, the class handles memory and the NSURLConnection delegate methods. As usual, the implemented methods are:

- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response;

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data;

- (void)connection:(NSURLConnection *)connection didFailWithError:(NSError *)error;

- (void)connectionDidFinishLoading:(NSURLConnection *)connection;

Any object willing to use FileDownload has 2 options:

1. Create the FileDownload object with initWithURL

2. Create the FileDownload object, and then start the download process using startDownloadFile

Sample implementation might be:

self.downloadObject = [[FileDownload alloc] init];

[self.downloadObject setJobID:theID];

[self.downloadObject startDownloadFile:theURL callBack:self];

Then just wait for the FileDownloadProtocol’s methods to be called.

This class is a Soot-and-Forget download, at least I call it that way, because once the download method is called, the application is free to do any other processing until the downloaded information is ready or a download error is raised.

I now use this class whenever I need to download data from a given URL.

In the next post of this series I'll talk about the FileDownloadController class.

My WebServices solution for iPhone

2010-06-15T12:00:00.000-07:00

I got into the necessity to call WebServices from within an iPhone application I was designing. After a small search with Google I found different implementations, even a “C” (or C++ I can’t recall) library, but nothing like the “easy way” used in Dot Net.

My specific application needs to show address book, kind of, information in a table. Each row shows the contact’s picture and name; when selected, in a second screen, detailed information will be show. This functionality is similar to the contacts' section of Facebook or LinkedIn application. The source information has to be gathered from a WebService.

Note: I will focus these articles in the WebService consumption, instead of the Graphical development and application logic.

For simplification, the WebService contract will be:
1.List screen
An XLM with up to 100 results with the next structure:
<contact>
<contactid>id for this item</contactid>
<photo>Url of contact’s photo</photo>
<name>contact’s name</name>
</contact>

2.Detail screen
An XML with contact’s detail:
<contactdetail>
<detail1>bunch of information</detail1>
<detail2>bunch of information</detail2>
<otherdetail1>bunch of information</otherdetail1>
...
</contactdetail>

Details might vary according with the available information, even tag names might change

In the first screen, information from first WebService Method is used to populate the table, the photo URL is use to download the image and draw it in the same table for each contact.

Once a row is selected, the second method is called with the ID of the contact, the retrieved information will be shown in a way similar to built in Contacts’ application.

First Stage was to develop a File Downloader class. This class helps me to download information of a given URL. With this class I test downloading a hardcoded XML simulating a call to first method.

Despite the fact I also build a generic all purpose XML Parser, which I might talk in a different post, the Second Stage was to think in downloading several files at a same time, to show the photos in the main contacts’ list. So, I build a Download Controller class to keep track of downloads and the objects requesting downloads. This controller uses the file Downloader build in First Stage.

In a Third Stage I create a WebService Controller class to support the call of registered WebServices. This class publishes the WebServices which will be called to retrieve information; it uses the Download Controller from Second Stage and, by consequence, the File Downloader from First Stage.

The final WebService Controller gave me an “easy way” to call WebServices from my iPhone applications as now, I only need to type down the Methods’ Firms in ordet to be available to use.

In the Next post I'll talk about the impementation stages.

Initial development projects at college

2010-05-27T12:00:00.000-07:00

As we were on a Computer related career, many classes required from us to develop a computer system. The most common language for development was FoxPro. Sometimes for graphical applications we used Visual Basic, but only for easy projects; I started using it from version 3. At that time MSDOS was the most common Operating System, but graphical applications were starting to get used.

Initial development courses

2010-05-13T12:00:00.000-07:00

At college we had 4 courses for development, those where more focused for us to had Language knowledge. Those classes were Pascal, GW-BASIC, COBOL and FoxPro, in that order. Programming Logic was an issue in those times as classes where focus to the language and not to the logic; also, not all of us had a computer at home, so, we had to visit the Lab for computer time. Time might not be too expensive, but availability was another issue.

At Pascal course, I had some problems with coding as I as to create big programs, most of them long IF Statements and the following of my core was hard to other persons, I took little attention to this issue until tow more courses. In a midterm exam, my teacher, Engineer Diana Margarita (Diana Margarita is guilty of my carrer), gave me a low grade for my code, I had to type it in the PC and show her the result to reconsider my grade.

Second course was GW-BASI, this course was given by Engineer Danilo, he had a “different” way to teach, but was good..

COBOL was challenging for the language itself and for the teacher, Engineer Zamora, he user to explain and write in 3 different whiteboards, when he finish the explanation we start “copying” to the computers for later execution. By these times, I was realizing about the importance of programming logic and I was writing better code. As I had a typewriting ability, I usually finish the coding before others, after finishing I walk the room watching others work, occasionally I try to help me classmates with their doubts.

FoxPro was a good course, it was teach be Engineer Juan Antonio Villarreal, we had very good ideas for teaching, almost always too different of other teachers. He use to talk about the “German Techniques” which were only intense use of the language to accomplish some tasks, usually it toke more than three commands in a single line which were only “translated” if you have an advance knowledge of the language; many were afraid of those techniques, to the point that were banned to some projects because of their complexity. Usually, the midterm exam was something like this:

Compute the 5% monthly tax needed to be deducted for all employees of a company. The period to compute is any given month. Read employees form a database. Each employee might have up to 2 salary raises in the period, these raises are stored also in the database. The only input is the month to compute, the exit is a report showing Employee number and name, total tax to be deducted.

When I first made this exam I used at lead 3 sheets of paper both sides. One day I was at the bus, going to college thinking in the exam and suddenly I realize how to do it in a better way; when I finish coding it, I was only 60 lines long, counting blank lines. This single exam and the former solution I did for it change the way I code for the future.

Before college

2010-04-29T12:00:00.000-07:00

My father spent many years as a Teacher in the Universidad Regiomontana (also see this) and he made an arrangement with computer lab to let me go and “practice” with the computers they had a t that time. I spent many hours, when the lab was free, programming in basic and playing in those computers. A game I remember is Karateka; at that time I was at High School and was taking some Karate classes of ShitoKai and that game took my attention.

At high school we had classes of BASIC, I previously used BASIC for my First big program, so this course was not quite exceptional for me, but I took some practice with the programs I created.

Shareware

2010-04-15T12:00:00.000-07:00

First Shareware application I recall was DOOM. As a shareware application it only came with a couple of free chapters and if you want to continue playing you must buy the full game. Shareware concept, from my point of view, revolutionize the software industry, from now on, applications came in a shareware mode, where some features will unlock until you buy the full application or if you don’t pay it, some functionality will be discontinued form the installed application.

iPhone and the copy and paste programming

2010-04-08T12:00:00.000-07:00

I was reading this post: Resize a UIImage the right way where is mentioned the copy and paste programming.

I agree this is a big problem when downloading code from the web without testing, but, copy and paste make things easier when developing similar functionality. The existence of code libraries makes us copy and paste code snippets to reproduce functionality.

Programs in the big picture are different, but in its internal functionality are similar. Database access is “barely” the same and don’t make sense to rewrite all the database access code, it is easier (and practical) to copy that code from a library or other sample we previously create.

At the bottom line, I really like the article and support copy and paste programming when doing the right way.

When one Diskette was enough

2010-04-01T12:00:00.000-07:00

When I start college, back in late 80, I start using more often the PC. In Mexico there were little penetration form PC for an average citizen; most computers were in Companies or Universities. As I decided to study a computer science engineering, part of the homework we were to do requires to be created using a PC.

When I start, the common storage was 5 1/4 inches Diskettes, the available storage was 360KB. Initially those were single sided, but with a notch on one edge you could turn the diskette in the reader to make if double sided, we can naw stored 720KB. Later, diskette reader evolved to read both sides without flipping the floppy disk.

Some common thing was to store all your stuff in the same diskette:

The Operative System (MS-DOS at that time).
Word Processor, a common software was ChiWriter I also remember having Professional Write at the same time.
Personal Files, sometimes you could have all homework from the year

Most of the computer had no hard drive and, consequently, no Operating System, that’s why we should carry on us.

A couple of years later, 3 1/2 inches diskettes appears and replace those big 5 1/4, those had more room for files and eventually replace them completely.

There were no concept of piracy in our minds, but we share programs among all of us to be in the edge of the technology at our reach.

Most of the common software those days had simple interfaces, but soon the graphical applications arrived.

My first big program

2009-10-29T12:00:00.000-07:00

Around 1990 my uncle Ibrahim and my aunt Enriqueta, gave me a great gift, a computer. I don’t remember the brand but it was a Commodore 64 equivalent, not really sure if it was a TRS-80. The Computer had Floppy Disk and also came with a Tape Recorder, and a couple of games in cassettes. Unfortunately the connection cable for the Tape Recorder was absent, but that was no problem for someone with initiative. After some experiments, and accidentally deletion of a couple of games, I made my own connectivity cable. The advantage of having the cable was that I was able to read and also save information in cassettes.

At that time I was already a fan of Robotech, so I start a personal project to create a Robotech Game. For the computer, the development language was BASIC. The Idea was simple, as with the series, the SDF1 will appears somewhere in the orbits of the outer planets and will need to return to earth.

That was a good project because I needed to make a lot of research. First I investigate a lot for the planets. Size, gravity, orbit, distance, year period, etc. I was trying to have a scale planetary system. The only thing I skipped was the elliptic orbits. All my planets had circular orbits. Once I had the orbits I included the spaceship, with the random start location. When I was remembering the project with a friend, he told me how I manage to get all that information together, specially at that time and without Internet; thinking backwards, maybe in the past we spend more time making research, but it gave us (at least to me) a satisfaction feeling of getting things done.

The game was easy, the ship will only move linearly, this is, it will not be orbiting around the Sun (there was no need for it as I want only to have fun), then with each step you indicate the acceleration and time span. As you accepted the values the computer make calculations and show the new position of the ship (and the planetary system). Sometimes you were able to see the planets in the screen. Also a zoom factor allows seeing all planets in the same screen or only a certain section of the system, always the SDF1 in the center of the screen. The goal was to get to the Earth, in pixels speaking, the SDF1 and the Earth need to be at the same position at the same time. This means we need to slow down as we get closer to Earth.

A subsecuent version of the game allows the SDF1 to move to any place in the solar system, moving not only in 1 axis. The game had random objectives, get to some planet because it needs our support. In this new version we need to stop exactly at the planet with a velocity of cero. The execution was also in turns. We first need to accelerate and at some point we need to slow down. Problem was that the planet was also moving, so, we need to anticipate its location for the time we get to it. For this problem the game had a suggestion option; given the required time, the game computes the final planet location and estimates the required acceleration/deceleration to accomplish the mission.

Again it requires some investigation to infer the required formulas to implement all options.

All programs were stored in cassettes using the Tape Recorder.

As I move on in technology, pass the computer to somebody else, but kept the good memories.

The beginning

2009-10-15T12:00:00.000-07:00

When I was a kid, around 1978, my father had the greatest job in the world: he was in charge of the Data Center of Vidriera Monterrey, well, we now call it Data Center, I don’t remember how they called back then, I suppose they call it the same.

Making some memory, I think the Job at Data Centers has been always too demanding. That’s why he didn’t like telephones at home, as it can be used to bring work to home (also at that time it was not so popular in México), for this reason, Vidriera often sent the company’s driver to pick him up when problems arise at work.

My father was a well known person in Vidriera, I remember going with him to the work, especially on weekends, and everybody greet him. We couldn’t walk more than a few steps without waving someone.

I remember many times when I got to the Data Center with those big rooms and big computers, tape readers and huge printers. The punched cards were strange sheets of paper and those my father gave us (to my sister and I myself) keep us busy trying to punch it all. My father show me once a system which stores employee information, I remember getting impress because the computer know my name and age.

I’m still proud of my father and having the opportunity of been his son. Once in elementary school, around 1984 the teacher invites my father to talk about his job, everybody was amaze about computers. Later on junior high, around 1985, I organized a group visit to Vidriera. Besides computers, the glass manufacturing process, with all that machinery, is amazing.

Around 1985, I went to a Children Computer Course at the Universidad de Monterrey (also look here), there I met the Apple II, a year later I went to another course at Facultad de Contaduria Publica y Administración where I learn a little about BASIC and worked with IBM compatible computers.

Having contact with those computers at an early stage of my life, specially at that time when computer were something only big companies can afford, made a change in my life; that was the beginning for me to decide to spend my life around computers.