اردوویب بلاگ

Some Windows XP users have been having the problem with UrduEditor that it does not properly join words. The solution is to properly configure the support for right-to-left languages in Windows XP. This is done as follows: Open Control Panel, go to the Regional and Language options. There select the Languages tab and check both check boxes as shown in the following figure.

Application of these settings will properly configure the required right-to-left language support on Windows XP. You will need to have your Windows XP CD ready. The needed files will be copied from the CD and at the end it requires you to restart the system. Now you can properly use UrduEditor. I am still trying to make UrduEditor work on Windows 98. It was difficult enough to compile the sources on Windows 98. The latest distributions of Visual Studio 6.0 and Platform SDK do not support Windows 98. I had to strip UrduEditor of the email functionality. Although it now compiles, it still does not yield the desirable results. I do not know how to enable support of right-to-left languages in Windows 98. I will thankful if somebody could guide me in this regard.

While the voting goes on about the domain name, let us consider the tasks we need to do once the domain is available.

The following sites need to be moved to the new domain:

1. This blog, Urdu Blogging: We need to switch the blog to Wordpress. Asif, can we use Wordpress 1.5 or should we stick to 1.2 for now?
2. Urdu blogs webring: In addition to moving the home page of the webring, I also want to switch it to some php/MySQL-based solution instead of Ringsurf.
3. Urdu Planet: This should be simple to move.
4. Urdu Wiki: Asif, we need to coordinate on that.

1. Urdu Freeware: In addition to Nabeel’s WebEditor and UrduEditor, is there anything else? What about Blogger, Wordpress and Movable Type templates optimized for Urdu blogs? Also, Qadeer, how about putting up the Urdu translation of phpBB here too? Does anyone want to set up a mirror to this?

   In addition to moving these to their new locations, we need to do it in a way that we don’t break existing links. So forwarding of each page (and not just the main page) needs to be properly set up at the same time. I would hope that this forwarding would stay up for at least a few months. However, the forwarding/redirect should not be completely transparent to the visitor as we want the visitor to know that the page has moved so that (s)he can update her/his bookmarks. The META Refresh tag seems appropriate for the purpose. The .htaccess Redirect also might work.

   If possible, it might also be a good idea to search for links to these items and inform the owners of those websites where the links are that our sites have moved.

   In addition to all the moves, I would also like to setup an Urdu forum. Qadeer, can you please help us there? Guys, any ideas of the design of forum topics?

   We would also need to maintain the main site as well as other static pages. A CMS would be really useful. Nabeel mentioned a few:

2. Mambo
3. Drupal

1. Typo3

   I have heard only of Drupal and that seemed too complicated to me. Also, we have to consider how much modification any of these would need to generate Urdu or bilingual pages. Are there any other CMSes out there that could be useful? If a CMS is too much work right now, how about using Wordpress or other blog software to generate static pages as well?

   Among ideas for the future, how about setting up an RSS aggregation of other Unicode Urdu websites (news etc.)? Any other ideas?

   Now comes the question of responsibility. Setting all of this up and maintaining it is going to require at least a few people. Therefore, please let me know if you are interested (I am looking at you, Asif, Danial and Nabeel and others too!) I’ll setup shell access accounts for everyone involved. Then we have to coordinate about who is doing what and who maintains what.

We are thinking of getting a separate domain for all the Urdu-related stuff we are doing. This would include the following:

• Urdu blogs webring
• Our group weblog about Urdu blogging
• The aggregation of Urdu blogs at Urdu Planet
• Urdu Wiki
• Urdu forum

• Urdu freeware like Nabeel’s Urdu Webpad and UrduEditor.

  Help us decide on a domain name by voting in the poll at Procrastination.

If anyone tried to comment in the last few days, I apologize for your comment not going through. When I upgraded to Movable Type 3.16 recently, I made a mistake so that any commenters were stuck at the “Preview” page and could not submit their comment. I have fixed it now. Comment away.

This post contains the following sections
Introduction
Using UrduEditor
Where is the on-screen keyboard?
Sending Urdu Emails
Posting to Yahoo and Google Groups
Development
Futher Development
Download
Feedback
References

Introduction
With this post I am presenting you the UrduEditor, a Notepad like application especially tailored for editing Urdu text. You can think of it as a souped-up Notepad, a stripped down Unipad or whatever. After developing the web-based Urdu editor (I will call it Urdu Webpad from now on) I thought of developing a native windows version of a similar utility which I call UrduEditor. Since then I have been working on this idea. The progress has been very slow since I have been extremely busy. I do not have time even to add and test the tiniest of features. There is still a lot of unfinished functionality. I have no choice but to release a version and declare it as beta. I am also providing the source code of the UrduEditor application under GPL. Everybody is encouraged to make changes and add features to the UrduEditor. I am also ready to change the license from GPL to some other license if somebody finds it too restrictive.

Using UrduEditor

The UrduEditor works in a way similar to that of Urdu Webpad. It is also capable of saving and loading Urdu text files. The Urdu text files are stored in UTF-8 format. Although the Urdu text files are plain text files, these are saved with a .uxt extension. This helps in identifying the Urdu text files in the windows file explorer and these also have shell association with the UrduEditor application i.e. double clicking on these files opens these in the UrduEditor application. The UrduEditor can even print, well, sort of. The problem is that the UrduEditor relies on the basic printing functionality of the MFC CEditView class. As a result the print-preview looks horrible and the printing is done left-aligned. I did not delve into the inner workings of the MFC/Windows printing mechanism. One solution would have been to use the CRichEditView class. That also would have meant a major re-write of the application. If there is anybody out there willing to augment the printing functionality of the UrduEditor application, they are most welcome to do it. One interesting feature of the UrduEditor is saving the Urdu text in html format. This html is of very primitive nature i.e. full of font tags and without any style. The reason why I chose this form of html was that I intended to use it for sending Urdu emails and this kind of html is most likely to be correctly displayed by any mail-client supporting html-mail.

At the moment UrduEditor uses the Urdu Naskh Asiatype font with a fixed point size. I will try to release a version soon that will allow selection of other Urdu fonts and point sizes. The keyboard mapping is similar to that of Urdu Webpad, which in turn mimics the keyboard mapping provided in the Inpage Urdu composing software.

I have also customized the Find and Replace dialogs to easily search and replace Urdu text. Figure 1 shows the main UrduEditor application window. Figure 2 illustrates the use of some toolbar buttons in UrduEditor.

Figure 1 : UrduEditor application

Figure 2 : UrduEditor’s Toolbar

Where is the on-screen keyboard?

This is probably the most glaring omission from the current release. I will try to provide an on-screen keyboard soon for aiding in entering Urdu text. Meanwhile the Windows 2000 and Windows XP users can use the on-screen keyboard provided with the standard windows installation.

Sending Urdu Emails

An editor sending emails? Well, why not? I had an idea about sending Urdu emails but was too lazy to develop an extra application for that. I found out that the most feasible way of sending Urdu text based emails is sending it as html-mail. There are arguments about the security concerns when receiving html-mail but I still think that it is indispensable.

I wanted to incorporate html-mail functionality without relying on the Microsoft Outlook’s object libraries. I looked around for a c++ class that could provide a thorough encapsulation of the SMTP-protocol. The best open source class that I found was CPJNSMTPConnection written by P.J. Naughter. In fact the send mail dialog is also stolen from the sample application provided with the above-mentioned class. Figure 3 shows the send mail dialog.

Figure 3 : Send mail dialog

Some configuration is necessary before actually sending mails. As shown in the figure, the mail server name and the port number has to be specified. The mail server can also be a POP-server. I have experimented connecting to an Exchange server and the POP-server provided by the webmail provider www.gmx.de. I have not been able to make it work with Gmail’s POP-server yet. There is an almost infinite number of combinations of the mail-clients and message formats and I have been able to test only a few. The configuration dialog is shown in Figure 4.

Figure 4 : Configuration dialog

I have successfully tested sending Urdu emails to Hotmail, Yahoo and Gmail accounts. Figures 5,6 & 7 respectively show the screenshots of Hotmail, Gmail and Yahoo mail with Urdu mail messages. This text is taken from some article at BBC’s Urdu website.

Figure 5 : Urdu message in Hotmail

Figure 6 : Urdu message in Gmail

Figure 7 : Urdu message in Yahoo Mail

It must be noted that you may or may not be able to freely choose the sender’s email address. For example I was able to specify any sender address I liked when sending email through an Exchange server but I was only able to send email through GMX after I specified my correct GMX-address. It is also interesting to note that when sending Urdu email this way to a Gmail-account, the email is likely to land into the spam-mail folder. Watch out for that!

Posting to Yahoo and Google Groups

One of my goals was to enable the members of Yahoo- and Google-groups to post in Urdu. This is possible by sending html-mails to the groups’ email addresses. The key to participating in a particular discussion is its title which must be specified as the subject of the sent mail. Following are the discussion groups that I set up for my experimentation:

Yahoo UrduProjects Group:
URL: http://groups.yahoo.com/group/urduprojects/
Post message: urduprojects@yahoogroups.com

Figure 8 : Urdu post in Yahoo Group

Google UrduProjects Group:
URL : http://groups-beta.google.com/group/urduprojects
Post message: urduprojects@googlegroups.com

Figure 9 : Urdu post in Google Group

It can be seen from the above images that the Urdu text is displayed better in the Google group. The text in Figures 8 & 9 was copied from a post in Danial’s blog.

Note: If you see garbage instead of Urdu text, change the default encoding of your browser to Unicode (UTF-8).

Development

I have developed the UrduEditor application using MFC and Visual C++ 6.0. I did not use Visual C++.NET because I wanted to avoid unnecessary dependencies that would prevent the program from running on Windows 98 systems. Asif has reported that the programs compiles fine with Visual C++.NET. I may gradually switch to Visual C++.NET for future development. The source code also presents an example of using Unicode with MFC. I have extended the MFC edit controls for editing of the Urdu text. I may at some later point create an MFC extension library useful for developing Urdu based applications.

Further development

A lot of work still remains to be done. I stay committed to the development of following features:

An on screen keyboard for helping in the entry of Urdu text.
Changing fonts and point sizes
More options for different html-templates
Dialog boxes for displaying and copying different types of html outputs
A toolbar for inserting diacritics

The ideas used in the development of the UrduEditor application can also be used in other directions. It is possible to develop a library of ActiveX controls to allow Visual Basic programmers to write Urdu based applications. The keyboard handling in the UrduEditor application can be made generic in order to support multiple keyboard layouts. This would make capable of editing text in local languages e.g. Sindhi, Balochi and Pashto. In fact I am already eying the keyboard layout format used in Unipad. I can accomplish this task only if I get enough support from people having interest in it.

Download

You can download the UrduEditor application and its source code from the following links.

Updated: Dowload UrduEditor with source code

Feedback

Your feedback is essential for the further development and improvement of the UrduEditor application. You can either post a comment on this blog or you can send a post to the above mentioned Yahoo or Google groups for your feedback.

References

1. CSS and Email, Kissing in a Tree by Mark Wyner, http://www.alistapart.com/articles/cssemail/
2. HTML Email and Using Style by Elizabeth Davies, http://css-discuss.incutio.com/?page=StyleInEmail
3. Adding Unicode support to an MFC-generated application in Visual C++ 6.0, http://www.differentpla.net/node/view/135

4. A helper class for reading, writing and appending to/from Unicode text files by PEK, http://www.codeproject.com/file/textfiledocument.asp

5. OfficeXP look and feel Owner drawn menu with the Office XP visual style by Jean-Michel LE FOL, http://www.codeproject.com/menu/menuxp2.asp
6. An MFC class to encapsulate the SMTP protocol http://www.naughter.com/smtp.html
7. Customizing the Common Find/Replace Dialog by Kalai Kandasamy, http://www.codeproject.com/richedit/customfind.asp
8. Microsoft Developers Network (MSDN), http://msdn.microsoft.com/library/default.asp

Tom Gewecke left a comment that he has put together an Urdu keyboard for Mac OS X. To download it, go to his webpage and download UrduPC.keylayout.

I don’t have a Mac, so I can’t check it out, but if you have one, try the keyboard out and let Tom know if there are any problems.

A lot of work still needs to be done there, but welcome to Urdu Planet. I hope you like it.

It is time again to share some thoughts with you but first there is some update on the Urdu Editor.

Update on Web-based Urdu Editor

I have fixed some bugs regarding punctuation marks and the full stop. The more significant update is that now the Urdu Editor also works in FireFox. You can download the updated version of the Urdu Editor from the same location.

Download UrduEditor
View demo page

Fixing the behaviour of the Urdu Editor in FireFox was an interesting experience. There is actually a JavaScript debugger for FireFox. This may not be very stable but it is still a huge improvement over popping up message boxes just to find out if something is going wrong.

I am grateful to everybody who liked the Urdu Editor and praised it. I am interested in finding out how many are actually using it i.e. the success stories kind of thing.

Accessible Urdu Font

Some people have told me that from the known Urdu fonts, only Urdu Naskh Asiatype works on Windows 98. Urdu Naskh Asiatype is a TrueType font whereas the Nafees font-family consists of OpenType fonts, which won’t work on Windows 98. I am not able to verify this because I do not have Windows 98 on my computer. But if it is true, we should at least offer Urdu Naskh Asiatype as an alternative font on our blogs and the UrduWiki so that it displays correctly also in Windows 98. The edit area of UrduWiki is set to use Tahoma which again is an OpenType font. Asif should have a look into this matter and make changes accordingly if needed.

The reason why I think Windows 98 is so important is its installed base. My assumption is that Windows 98 users still constitute the majority of computer users in Pakistan and India. We surely don’t want to shut ourselves off from the very audience that we are trying to attend.

Blogging in Local Languages

It is heartening to see the growing number of Urdu bloggers. Some bloggers like Asif occasionally post a blog in Punjabi. I think we should encourage this trend. Blogging can be a way to promoting Sindhi, Balochi, Punjabi and Pashto on the net in the same way it is being for Urdu now.

The web-based Urdu Editor can be very easily customized for typing in Sindhi, Balochi etc. One needs an appropriate font that contains the glyphs for the characters in some particular local language and the information about keyboard mapping. If there are people who need help in this regard, they can contact me through this blog or at simunaqv [ at ] gmail [ dot ] com.

Why is having a HTML-based Urdu editor so important? After all you could just set a few attributes of the HTML textarea element and you could start typing away both Urdu and English. The input language changes with the keyboard layout selected. As it turns out, this arrangement is not available to the Windows 98 users who still constitute the majority of computer users in Pakistan and probably in India too. The web-based Urdu Pad created by Urdu Life has proved highly popular in the Urdu community. Even many Windows XP and Windows 2000 users use Urdu Pad because of its easy to use and intuitive interface. You do not have to switch keyboard layouts. Instead you can just toggle the input language using the radio buttons. The key mappings, which are similar to those provided by the Urdu phonetic keyboard, are said to be similar to those present in the InPage Urdu composing system.

All of this is very nice until you notice that the people at urdulife.com do not allow the use of the Urdu Pad outside the premises of urdulife.com. That means you cannot integrate this as an input component in your web applications. Urdu Life also safeguards its intellectual property by obfuscating the JavaScript source code of the Urdu Pad. Though it is true that every business has the right to protect its intellectual property, we need more open source way of thinking if we want to promote Urdu. While open source is thriving in the rest of the world, Urdu is still struggling to get out of the stranglehold of closed standards and proprietary software.

Seeing the need for a web-based Urdu editor component that could match in functionality to Urdu Pad, I decided to develop one myself. Considering that I am a novice in JavaScript programming, this task was somewhat daunting. I had set out several goals before starting this development, not all of which I could meet. I wanted to make this a reusable component. I have been partially successful in achieving that goal. The work on Urdu Editor is not finished. I hope that the Urdu community takes interest in this project and contributes to it.

Compatibility Issues

I faced a setback when trying to make the Urdu Editor component compatible with FireFox. It is working only with Internet Explorer at the moment. The working of Urdu Editor is simple. It inspects the key codes in the keypress event and based on the language setting modifies this key code. The event model in FireFox does not allow this alteration of key codes. Maybe there is some workaround for making it work but since this exploration was taking so much time, I gave up on it. I would request my friends with JavaScript expertise to inform me of any workaround, if there is one.

Licence

I am distributing the Urdu Editor component under GPL.

Use

The Urdu Editor is different from Urdu Pad in that you can place as many single-line and multi-line Urdu enabled edit areas on your web page. The keyboard control is not tightly coupled with any one specific edit area. It sends characters to the edit area which currently has focus.

Include the following line in you HTML code before you can use the Urdu Editor:

where yourpath is the path to the JavaScript file UrduEditor.js. Now you have to call the initUrduEditor function as shown in the following.

As is shown by the semantics, cssPath is the relative path to the UrduEditor.css file. The function initUrduEditor initialises some variables and event handlers.

After the necessary initialisation, there are more than one ways to create an Urdu edit area in you web page. One possibility is to call the writeUrduEitor function. This function has the following semantics:

id is the identifier assigned to the edit control. The rows parameter determines height of the edit control. An input element of type text is placed if rows is equal to one, otherwise a multi-line textarea is created. The width of edit control is determined by the cols parameter and the pointsize parameter sets, well, the point size of the edit area. The Urdu Editor uses the Urdu Naskh Asiatype font. Use of this font makes sure that Urdu Editor also works in Windows 98.

There are situations where you may want to customize the edit areas of existing web pages. An existing edit area can be converted to an Urdu Editor control by carrying out the following steps:

1. Assign an id to the edit control if it already does not have one
2. Set the attributes lang="ur" and dir="rtl"
3. Apply the appropriate style e.g. style="font-family:Urdu Naskh Asiatype; font-size:16"
4. set the event handler, onfocus="setEditor(this)"
5. set the keypress event handler, onkeypress="processKeypresses()"
6. set the event handlers onclick="storeCaret(this)" onkeyup="storeCaret(this)"

I have provided a function makeUrduEditor(id, pointsize) which does almost all of the above for an edit control identified by id. Strangely enough, setting the onfocus event does not work with makeUrduEditor. Therefore onfocus must be manually set as described above.

The keyboard control has a table-based layout. As you can see from the source code, it expands or shrinks to fit the space provided. How the keyboard control appears on screen and how easily it is accessible for all edit areas depends on how the web page was designed. The toggle language buttons are also part of the keyboard control. Calling the writeKeyboard function creates the keyboard control.

Example: Integration into phpBB

Our friend Qadeer was especially getting impatient to get his hands on the Urdu Editor because he wanted to integrate it with phpBB. In his opinion such a control would go a long way in promoting Urdu on the net. We have to wait and see if he his prognosis is right. I successfully integrated the Urdu Editor inside phpBB’s post page. I am including the modified template file posting_body.tpl in the download. First of all I copied the files UrduEditor.js and UrduEditor.css into the folder templates/subSilver/ and then added the reference to UrduEditor.js to the posting_body.tpl template file. Following shows the inclusion of the reference to the UrduEditor.js file and the call to the initUrduEditor function.

There are two single-line edit controls and one textarea element on the posting page. The single line edit control correspond to the user name and the subject fields and hence are assigned name values of “username” and “subject”. As described above, I assigned id’s to both edit fields which were the same as their names. I also added the onfocus event handler. The final touches are given by the makeUrduEditor function call as shown in the following:

and

The multi-line textarea element receives the actual message post text. The only customisation required in this case is the assignment of the id as well as the setting of the event handlers onfocus and onkeypress.

The elements on the post page are arranged using a table-based layout. I added another table row and a table cell where I inserted the keyboard control using the follwing code:

The following figure shows the results of the customisation of the post page.

Download UrduEditor
View demo page

Enhancements

There is still lots of room for improvement. Possible improvements include opening the keyboard control in a separate modeless dialog, a better color scheme of the keyboard layout, adding more key codes and diacritics etc. You are most welcome to take the provided source code apart and introduce enhancements. My request is just to notify me of any new feature that you add or make it yourself available to the others.

I was waiting for the final release of Wordpress 1.5 to work on my Urdu theme. Here is my Urdu blog, I still need to modify a few files and add some javascript buttons in my comments form. Please have a look and let me know what you think of it. This theme has different templates for categories, individual posts, comments, search, sidebar, links etc. I will soon add it in Urdu Wiki with installation instructions and a wp-admin.css to change the font in wp-admin section.

There are some bugs in this release of Wordpress, like mt-import doesn’t work at all. Best way to move a movabletype blog to wordpress is to install wordpress 1.2, run the import-mt.php and then upgrade to 1.5.

I had a little discussion with Nabeel bhai about search engine optimization for Urdu blogs and webpages. I think that we don’t need to worry about it because if there is content somewhere then Google is quite good at indexing it and including it in relevant results. Like this Urdu blog is listed in Google while Urdu wiki is not yet indexed. The reason is not that we have something missing there in Urdu Wiki but its the fact that Google gives priority to blogs. Their bot totally ignores meta tags such as keywords and description. They also ignore the repetetive keywords in the body tags.

Their pagerank system works on how many people link to you and with what text. Like if you search for Danial my blog appears at the top. Search for Urdu blog and Salambazar’s Urdru blog appears at the top, search for Urdu and BBC Urdu tops the results. Why BBC Urdu tops the result? Because people are linking back to every thing they have with the keyword Urdu in it.

I liked the idea of offering cute little graphics for publicity of Urdu Wiki. Why people offer these tiny gif files? Because that way you can make people add a link back to your site using your own words.