I used pure relational database approach (postgresql to be exact) to store information collected via flickr initially, but it wouldn’t really scale (or I just simply suck at designing/maintaining database tables) and my queries got killed numerously by the web host. Besides that, as the table grow larger, the amount of resource consumed to serve a query also grow, hence I needed to find another way to store the data. At first I thought of using some popular noSQL solutions, but my supervisor told me to turn the data into RDF format instead. So I went on with Redland and use postgresql (again) as storage.

However, for some reason, postgresql doesn’t seem to work efficiently and a simple query can take tens of minutes to run when it holds more than 100k RDF statements. For some reason, the hash storage doesn’t work on my Ubuntu development VM, so I switched the storage engine to MySQL after @dajobe’s suggestion.

However, as PHP binding apparently not that popular, so there aren’t much information / tutorial posted. I actually wanted to write a collection of scripts to collect data from flickr for my research project, so I began with finding an existing script for that task. However, I didn’t find good ones in PHP, so I ported this from Perl to PHP and use Redland. Then after knowing how it really works, I rewrote everything again from scratch (will put them up to bitbucket when I have time to clean up). I even wrapped the library with class methods, which I hope to release later (I only knew about another OO wrapper in PHP for Redland after almost done writing my scripts).

To use Redland’s PHP functions without OO wrapper, we always start with a statement to build a world. I am not very sure what this means (yes, I didn’t really read the documentation that thoroughly), but it seems that almost all constructor functions depend on it to create new object (yes, Redland has this OO feel although all the function calls are in procedural style). So, in PHP, the statement would look like:

$world = librdf_new_world();

Then you would want to decide where to store your RDF statements. For this piece of note, I will just use non-persistent memory store. So the function call to build a storage object is

$storage = librdf_new_storage($world, 'memory', $name, $options);

where $name stores the name of the storage object, and $options often carries the DSN, but in our case it is NULL. Now that we have storage defined, then we proceed with building a model to actually store RDF statements into the storage (it would be easy to think model as a database library, and storage as an abstraction layer).

$model = librdf_new_model($world, $storage, NULL);

Statements are consists of nodes, so let’s create some. To create a URI node, it is just as easy as

$foo = librdf_new_node_from_uri_string($world, 'urn:foo');
$bar = librdf_new_node_from_uri_string($world, 'urn:bar');

Creating a literal node would be just as simple as

$baz = librdf_new_node_from_literal($world, 'baz', NULL, FALSE);

Putting them into a statement

$statement = librdf_new_statement_from_nodes($world, $foo, $bar, $baz);

which is the equivalent to this

<urn:foo> <urn:bar> 'baz' .

To run a query to the model, just simply send a SPARQL statement as follows

$query = librdf_new_query(
    $world,
    'sparql',
    NULL,
<< ?object .
        }
SPARQL
);

Running the query and get result

$result = librdf_model_query_execute($model, $query);
var_dump(librdf_query_results_to_string2($result, 'json', 'application/json', NULL, NULL));

Enjoy!

Users of Web 2.0 applications typically organize their created content with a set of terms or keywords, also commonly known as tags ^{[3, 4, 6, 7]}. Spending additional time and effort to provide annotation such as tags to a resource usually implicitly implies the relevance and/or importance of the resource to a user ^[1]. Therefore, besides being used for content classification, tags assignment can also be analyze to find the implicit relationship between users and the tagged content.

It is also observed that the frequency of term usage is proportional to the level of importance or connection to user, i.e. the more a tag is used, the more it is important to the user ^{[1, 8]}. The choice of tags when annotating a resource often express the user interest and perceptions ^[8]. Users usually have different ideas and interests when tagging a resource especially when the application allow the same resource tagged by multiple users ^[2]. Therefore tagging activity pattern can be studied to find a user’s domain of interest.
Users with common interests tend to use similar set of tags in tagging re- sources of interest ^[2]. The similarity of users can then be calculated through the tagging pattern for recommending purpose as it provides social relationships between users ^{[4, 7]}. This may allow more collaboration between users sharing similar interest.

Although the variety selection of tag helps in better representation of the resource, but this may cause some problems as users don’t tend to agree to each other ^[2]. Observations also shows tagging behaviour follows power law distribution where tags are mostly assigned to a small subset of more
popular resources ^{[2, 7]}. Hence, when a resource gets popular, users having different interests will attempt to assign tags with keywords from all different domains. However, when different tags that are semantically related from each other are assigned to related resources (vocabulary mismatch problem), it may drastically decrease the level of relevance although the tags may actually share similar meaning ^{[3, 7]}.

Correlation between resources is often established when they are assigned with common tags ^[8]. The level of relevance can then be observed by studying the similarity in their set of assigned tags ^[8]. The correlation and similarity between the tags assigned to resources may also help users to find unseen resources by browsing the tags.

The assigned tags can be used as a tool to enable discovery, sharing and collaboration of web resources ^{[1, 2, 3, 8]}. Some applications even allow sharing of tagging information ^{[2, 8]}. Hence, besides using tags as a tool for content classification and user interest study, they can be used to make collaboration easier.

A resource may be described by its assigned tags depending on the tagging behaviour ^[1]. When an tagging system follows the bag-model, i.e. system that allows multiple users to tag a resource, the frequency of a tag is assigned also indirectly shows the increased relevance between the tag and the resource ^{[1, 2, 8]}. Weights can be also calculated for each tag to rank the relevance with the resource.
On the other hand, systems that implements the set-model do not provide enough data to deduce the level of importance or popularity of an assigned tag to the resource ^[8]. Due to the unstructured nature of collabo- rative tagging, the system can be easily abused if users assign tags that do not describe the content ^{[1, 8]}. The problem is more apparent for system implementing the set-model as there is no way to rank the assigned tags.

References

1. I. Cantador, A. Bellog ́ın, and D. Vallet. Content-based Recommendation in Social Tagging Systems. Methodology, pages 237–240, 2010.
2. Y. Guo and J. B. Joshi. Topic-based personalized recommendation for collaborative tagging system. Number 10. ACM Press, New York, New York, USA, 2010.
3. R. Lambiotte and M. Ausloos. Collaborative tagging as a tripartite network, 2005.
4. P. Mika. Ontologies are us: A unified model of social networks and semantics. Web Semantics Science Services and Agents on the World Wide Web, 5(1):5–15, 2007.
5. C. Schmitz, A. Hotho, R. J ̈aschke, and G. Stumme. Mining Association Rules in Folksonomies, pages 261–270. Studies in Classification, Data Analysis, and Knowledge Organization. Springer, 2006.
6. N. Shadbolt, T. Berners-Lee, and W. Hall. The Semantic Web Revisited. IEEE Intelligent Systems, 21(3):96–101, May 2006.
7. S. Siersdorfer and S. Sizov. Social recommender systems for web 2.0 folksonomies. ACM Press, New York, New York, USA, 2009.
8. M. Szomszor, C. Cattuto, H. Alani, K. OHara, A. Baldassarri, V. Loreto, and V. D. P. Servedio. Folksonomies, the Semantic Web, and Movie Recommendation. eprintsecssotonacuk, pages 71–84, 2007.

Long story short, I have been working on my research project two months ago. Throughout the mentioned time period, I basically spent a lot of time trying to read and understand some of the related topics in semantic web, information retrieval as well as machine learning as suggested by my supervisor.

I first started with revisiting RDF by putting more emphasis on how it is usually implemented. Yea, I know I did some when I was preparing my research proposal, but I didn’t go to the detail and skipped the section describing RDF/XML all together.

Recently there have been some debates on whether JSON should be used as another data serialization format. I personally like JSON for its simplicity (especially comparing it to XML), but I find RDF/XML actually makes sense (read: good enough) because I find RDF is very URI dependent and it would probably be a bit awkward to see a lot of URI popping up in JSON. However, I am still very interested to see how this is sorted out, as W3C initiated a group to work on it. (Some thought-provoking read on this by nathan@webr3)

I have also read on RDFS for the first time and the more I read it, the more I find it related to my classes on AI (especially Prolog) in my Bachelor Degree. However, I am also slightly surprised to see how tolerant it is (leaving too much room for error?). Besides RDFS, I also read on OWL and find more similarity with Prolog (I really should post the summary of the readings here, yeah, too much to blog about, but lack of time and motivation).

What really surprised me was that with the combination of these (RDF, RDFS and OWL) technologies, I actually tried re-inventing the same thing for my last task in my previous job (over and over again). Although usage of RDF may not exactly fit the problem, but I feel it is close enough to solve the problem.

While looking for some articles on Linked Data Principle, I came across this TED talk by Sir Tim-Berners-Lee. Yeah, I do feel sorry for not publishing enough semantic data online after watching that, the talk was very motivating indeed (but is posting personal info that is machine readable without limit actually a good idea, I don’t feel like injecting a great portion of my life into social networking site, but that’s a different story).

I also read the beginner’s guide to SPARQL, and this is the real wow-factor to me. Although I don’t really like the SQL-like syntax, but the similarity to Prolog makes everything looks cool now. Yeah, I am pretty new in this, so I keep referring everything to Prolog XD.

Then I started to read on information retrieval, this is when everything suddenly seems not cool anymore. However, it did force me to do a lot of revision on statistics and some basic maths stuff (you will be surprised to see how much I forget after working for 2 years). And it wasn’t encouraging when I began to watch a video lecture on machine learning hoping to pick up in shorter time span because I find myself not understanding most of the lectures XD.

Yeah, that’s basically wraps up my progress for the past 2 months, getting myself excited with new stuff for the first month, and getting frustrated in the second month not able to understand the material.

Besides, it is also a principle of linking data between systems and entities that allow rich self-describing inter-relations of available data across the globe on the web. Web of Data also marks a shift from publishing data in human readable HTML documents to machine readable documents that allows machines to be able to make inference out of the data published.

To efficiently link entities together, Sir Tim Bernes-Lee proposed four rules or expectations, as follows,

• Use URI as names for things
• Use HTTP URIs to enable people to look up these names
• Returns useful information using standard technology / format like RDF / SPARQL when someone looks up a URI
• Allow users to discover more things by including links to other URI.

However, it is important to know that breaking the rules does not neccessary destructive. It would only reduce inter-connectivity, which in turns discourages re-usability that results in making resources less valuable.

To use URI as names for things, Universal URI set of symbols should always be used to enable other parties to be able to process data that results in a consistent result. This also means that the risk of loosing meaning is reduced in the process.

It is also important to actually serve information on the web against a given URI. This allows data as well as metadata in specific standard formats such as RDF or OWL accessible. By publishing information about a resource, it enables others, especially applications and machines to properly understand the document.

To enable users to discover things, one of the ways is to provide inner as well as outer links information back to the user. By definition, it says that given a graph G, it is browsable if for the URI of any node in G, and if the URI is looked up, information returned that describes the node must satisfy the following conditions:

• Returns statements where the node is either a subject or object
• Describes all blank nodes attached to the node by one arc.

In short, it allows data to be represented in graph form and allow traversal. It is important for the query service to return RDF statements of that involves the specified node regardless it is a subject or an object. Note that the subgraph returned should be a minimum spanning tree (MSG) or known as RDF molecule.

However, if there is a statement that relate multiple entities, the statement should then be repeated for each of the entities. This then violates the rule where data must not store in more than one place mainly for consistency purpose. However, this becomes less of a concern if the statements are automatically generated.

Besides that, there may be a situation where the author of document A claims that it relates to document B, but the author of B may think otherwise. One of the reasons may be document A did not exist when B got published.

Multiple or expired data may also not be desirable at times. For example page visitor statistics data within a site introduction document. To solve this issue, one of the proposed way is to separate the statistics statement data out of the introduction document into an individual document.

In the end, links opens up the web of data to not only human beings, but also AI processes to allow them to make inferences out of entities. Besides, it also encourages all parties to publsih data freely in an open standard format.

Data summarized and greatly simplified from the following sources:

• Sir Tim Bernes-Lee – Linked Data – Design Principle
• Linked Data

The main objective of RDF is to generate information that can be processed by applications by defining a standardized approach to represent resources. The usage of standardized language also enables interchanging of information between applications without loss of meaning. This allows third party applications to use / consume information created and because the information format is standardized, tools are readily available to manipulate the information.

As mentioned earlier, as long as a thing can be represented in the form of Universal Resource Identifier, URI, then it can be represented / described by RDF. URI is generally used to represent not only network-accessible things, but also non-network accessible things like a arbitrary human being, corporation, or even a book in a library as well as abstract concepts that do not necessarily exist physically like creator / author / modified date. URL, which stands for Uniform Resource Locator is a subset of URI.

RDF is a simple language that deals with only binary relationship, which involves a Subject, a Predicate and a Object. Given an example “web(cslai) is Jeffrey04′s blog”, we can re-structure the statement into subject = web(cslai), predicate = owner, object = Jeffrey04. Then we can put this into a graph (kinda reminds me of Semantic Network), as follows (not RDF graph):

From the graph, we can see that the relationship between a subject and object is described by the predicate. Or another way of saying, subject ( web(cslai) ) has a property in the form of predicate (owner) that has a value of object ( Jeffrey04 ).

Before going to construct a RDF graph, it is important to know that as RDF is used to provide information on a resource, there are a set of basic rules to follow when constructing a RDF statement. Subject should always be a URI, or a blank node (will be discussed later) that denotes a resource, predicate must always be a URI and object can be another resource, a blank node or a constant represented by a character string.

Besides serializing a RDF graph into an XML file, the statements can also be written in the form of triples. Each statement in a graph is written as a simple triple of subject, predicate and object in exact order. Another point to note is that a graph is a primary manner of represent statement, and any other way to represent a statement is considered secondary.

The basic syntax of a triple requires URI to be enclosed in angle bracket or QNAME which kinda resembles XML vocabulary/namespace thingy, and literals to be enclosed in double quotes. For example

<http://cslai.coolsilon.com/> csterms:owner "Jeffrey04".

Blank node is introduced when a structured data presents as the object value. For example, given a triple as follows

exstaff85740 exterms:address "1501 Grant Avenue, Bedford, Massachusetts 01730"

Before discussing the graph, it is worth pointing out that each URI node is denoted by an ellipse, and literals denoted by a box. As seen on the graph, all the nodes are either a subject or an object while arcs are predicates. The respective RDF triples for the above graph are shown as follows:

exstaff:85740     exterms:address    exaddressid:85740 .
exaddressid:85740 exterms:street     "1501 Grant Avenue" .
exaddressid:85740 exterms:city       "Bedford" .
exaddressid:85740 exterms:state      "Massachusette" .
exaddressid:85740 exterms:postalCode "01730" .

As seen in both the graph as well as the RDF triples, a new node is created just to describe the concept of address. To represent the same piece of information in another way without having to create a new node, a blank node can be introduced, as follows

In RDF triples form

exstaff:85740 exterms:address    ??? .
???           exterms:street     "1501 Grant Avenue" .
???           exterms:city       "Bedford" .
???           exterms:state      "Massachusette" .
???           exterms:postalCode "01730" .

As seen from the graph, address node is changed from a node with URI address into a node that doesn’t have address which is called a blank node. Then in the triplets it is written as a ‘???’ instead of the complete URI as shown in the above example. However, besides using a ‘???’ to denote a blank node in triples, we can also use another form of representation in case there are a lot of blank nodes that represents different things within a graph. By reusing the same example, the triples can be rephrased as follows

exstaff:85740 exterms:address    _:johnaddress .
_:johnaddress exterms:street     "1501 Grant Avenue" .
_:johnaddress exterms:city       "Bedford" .
_:johnaddress exterms:state      "Massachusette" .
_:johnaddress exterms:postalCode "01730" .

By breaking up the address into smaller structured parts, it enables external applications to manipulate the information in a more standardized way to produce a more predictable result. However, because RDF only deals with binary relationship, to properly describes N-ary relationship, it has to be broken into a list of binary relationship with the use of blank nodes. Somehow, this reminds me of something similar in prolog where you can create a kind of variable that the programmer do not need to explicitly name them.

Besides being used for the above situation, blank node is also often used in situation where there is no other way to properly and accurately describe a resource. For example, a person with email address johndoe@example.com, although mailto:johndoe@example.com is a valid URI, but as mailbox address is also used as an attribute, the better way of representing John Doe is by using a blank node with mailto:johndoe@example.com as object, as follows:

_:john exterms:mailbox <mailto:johndoe@example.com> .

An example of combining multiple RDF that is scattered around the internet is assuming there is a book that is authored by an author that uses johndoe@example.com, we can make inference by comparing the following triple

ex2terms:book78354 exterms:mailbox <mailto:johndoe@example.com> .

We can then deduce that The book is written by John Doe that has email address of johndoe@example.com.

However, because by default object allows any literals, it may make other application that consumes the information in trouble. For example, by looking at the triple below,

_:jeff exterms:age "24" .

There is no way to tell whether that 24 is a base 10 decimal, or is an octal number. Things can only go worse if the application is actually expecting a float. Type literal is introduced to solve the problem by allowing the specification of a particular datatype to be used in literals. Back to the previous example, to properly define my age, I should prepare a triple as follows:

_:jeff exterms:age "24"^^xsd:integer .

Content is summarized / oversimplified from RDF Primer.

Semantic Web

Just thought it would be better if I start by stating a problem, imagine one day Alice is looking for the price for a piece of movie DVD, what she would do is to get to the movie store website and search for the movie DVD. However, Alice might not have been able to get the price if she is not around as the task cannot be automated easily using a computer. This is because the web page that displays the price is prepared for human being like us.

Therefore semantic web is a proposed solution to the problem where information should be represented not only for human being like us to read, but also for machines to be able to understand and manipulate it. In short, the definition says that Semantic Web is an extension to the World Wide Web, which information is given well-defined meaning that enable both lay person and computers to work in co-operation.

Representing Information

Semantic HTML

From what I remember, before Google emerge as the most popular search engine, webmasters used to include descriptive meta tags within a HTML document, as follows:


<meta name="keywords" content="good looking, handsome, single">
<meta name="description" content="About Jeffrey04">
<meta name="author" content="Jeffrey04">


Such that, when user search with the keywords as specified above, the document above will be returned as one of the search results. However, although metadata is displayed via meta tags as mentioned above, it is never enough to enable computers to understand that “Jeffrey04 is staying in Malaysia” or “Jeffrey04 works in Kuala Lumpur”.

Then CSS got popular as people around starts encouraging separation of content and presentation. Therefore, more people start using HTML tags like <strong> instead of <b> because <b> doesn’t carry any meaning. Usage of tags with semantic meaning also enable users that relies on screen-reader to further understanding the material.

However, often times especially while HTML5 is still under drafting we group content into sections enclosed within a <div> tag. For formatting purpose, as well as giving the block of information meaningful to machines, a class name is often given to the block, eg. <div class="header">. The usage of tag attribute to give meaning to a piece of information leads to the development of various microformats.

Microformats

When web-developers start using tags that accurately representing information within a document, numerous efforts are made to further mark up a document according to standard to ease machine processing. If one have ever does screen-scraping, they will feel the pain of trying to make the script to scrap the right information out of a HTML document.

To enable information to be read easily out of a HTML document, what a web-developer can do is to mark up the information following the specific standard. One of the standards is hCalendar, which is used to describe dated event, for example to mark up an event that is taking place at 6th June 2010, we would do:


<p class="vevent">John Doe is <span class="summary">getting married</span> on 6th June 2010 at the <span class="location">Community Hall</span>. The ceremony will be held from <abbr class="dtstart" title="2010-06-06T14:00:00+08:00">2PM</abbr> till <abbr class="dtend" title="2010-06-06T16:00:00+08:00">4PM</abbr>.</p>


Therefore, to do screen-scraping, one would just need to search for (via either CSS or XPath) the particular block of content above.

(to be continued, next is on Resource Description Framework, RDF, which loosely-related to my previous post on semantic net as linked above)

What is Semantic Network

As far as I know, a semantic network is a type of knowledge representation. A mind map, IMO, is a type of network diagram which loosely based on semantic network. Back to the topic, a semantic network is usually drawn to represent some knowledge through interconnecting nodes with labeled arcs. One of the objectives in drawing a semantic network is to do inferencing by referring to the interconnecting nodes. (I will post diagram once I find my e-copy of the lecture slides to illustrate)

Because my lecturer discussed about Prolog before moving on to semantic network, so we discuss semantic network in conjunction with Prolog way to represent facts. The steps taken to produce a semantic network for the fact “Mary is a lecturer” may be as follows:

1. Change the fact into Prolog fact: lecturer(mary)
2. Rewrite the prolog fact into another form: instance(mary, lecturer)
3. Now draw a semantic network with two nodes – mary and lecturer.
4. Then connect the two nodes with an arc labeled instance (diagrams coming soon)

Of course the above produced network is simple because the given fact is simple in the first place. However, when there are more than two arities (predicates) in the prolog fact then the semantic network can be reasonably complex as follows.

My lecturer discussed the inferencing process with an example that uses multiple related facts. For example, we have two person, Peter and Jane each has the height of 180cm and 177cm. The initial semantic network may seems as follows:

To do inferencing, a special procedure is added to the network to process the nodes. Without the special procedure then the analysis of the diagram would become limited.

As you can see the semantic network got expanded and an arc is added to link the previously unrelated facts. Hence, we can make an inference through observing the diagram that Peter has the height of 180cm greater than Jane’s at 177cm.

Then the semantic network can be expanded as complex as possible in anyway one may think logical. Of course there may be more to cover in this topic like expanding and partitioning the semantic network, using frames to overcome the limitations of semantic network etc. But I would just stop here to check whether I have made any mistake before moving on.

Further reading: Semantic Network by John F. Sowa