Vazaar Project Blog » Workshop

New custom timeline widget

Thu, 02 Sep 2010 02:15:14 +0000

Tired of looking for a timeline widget in PyGTK for Vazaar I’ve decided to develop it by myself.

It’s very simple: 1 calendar and 3 treeviews.

The user clicks a day in the calendar and with the selected date Vazaar asks to the database for those resources created that day, the day before and the day after.

Here is a screenshot:

You can find an example in the repository.Similar Posts:

Attached Files:

Vazaar Timeline
A timeline for vazaar showing resources for selected, previous and next day.

posted in GUI by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

How to fix your sqlite database after corruption

Wed, 01 Sep 2010 23:45:00 +0000

If you get something like this:

Traceback (most recent call last):
  File "./bin/vazaar", line 79, in
    Vazaar(options, args)
  File "/home/t00m/dev/trunk/vazaarlib/main.py", line 61, in __init__
    self.store = StoreManager(self)
  File "/home/t00m/dev/trunk/vazaarlib/store.py", line 54, in __init__
    self.__get_graph()
  File "/home/t00m/dev/trunk/vazaarlib/store.py", line 85, in __get_graph
    self.graph.store.open(self.options)
  File "/usr/lib/pymodules/python2.6/rdflib/store/Sleepycat.py", line 77, in open
    index.open(index_name, dbname, dbtype, dbopenflags|db.DB_CREATE, dbmode)

MemoryError: (12, 'Cannot allocate memory -- Lock table is out of available locker entries')

Just go to database directory and delete all files similar to __db.

Reason:
This problem occurs when Vazaar accesses the Berkeley database files, it makes temporary locker entries within the tables while it searches for data. If Vazaar crashes often, this issue will occur much sooner because the locks are never cleared.

Tip found at http://www.linuxmaza.com/system-administration/solved-lock-table-is-out-of-available-locker-entries-yum-install-yum-upgrade-3/Similar Posts:

posted in Python by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

PyGTK XDot Widget

Thu, 19 Aug 2010 15:15:54 +0000

XDot es un magnífico componente para representar grafos interactivos con Python, Graphviz y GTK.

De la web del autor:

xdot.py is an interactive viewer for graphs written in Graphviz’s dot language.

It uses internally the graphviz’s xdot output format as an intermediate format, and PyGTK and Cairo for rendering.
xdot.py can be used either as a standalone application from command line, or as a library embedded in your python application.

Features

Since it doesn’t use bitmaps it is fast and has a small memory footprint.

Arbitrary zoom.

Keyboard/mouse navigation.

Supports events on the nodes with URLs.

Animated jumping between nodes.

Highlights node/edge under mouse.

Este código ilustra como usar el widget en una aplicación. En la mía, genero dinámicamente el código dot según los nodos (recursos) que haya eligido el usuario. En los adjuntos puedes encontrar otro ejemplo mezclando rdflib y xdot:

import gtk # gtk bindings
import xdot # xdot library

class FooXDot(xdot.DotWidget):
def __init__(self, func):
xdot.DotWidget.__init__(self)

# Filters allowed: ‘dot’, ‘neato’, ‘twopi’, ‘circo’, ‘fdp’
self.set_filter(‘circo’)
self.connect(‘clicked’, func)

def display_dot(self):
# Header
dc_head = """digraph Vazaar {\ncharset="UTF-8";\n"""

# Relations
dc_body = """"user1" -> "tennis";\n"""
dc_body = """"user1" -> "basket";\n"""
dc_body += """"user1" -> "soccer";\n"""
dc_body += """"user2" -> "basket";\n"""
dc_body += """"user2" -> "soccer";\n"""
dc_body += """"user3" -> "tennis";\n"""
dc_body += """"user3" -> "soccer";\n"""
dc_body += """"user4" -> "karate";\n"""
dc_body += """"user4" -> "soccer";\n"""
dc_body += """"user4" -> "tennis";\n"""

# Key Nodes
# people
dc_body += """"user1" [URL="%s", url="%s", label="%s", shape="box", \
style="filled", fillcolor="yellow"];\n""" % (‘user1′, ‘user1′, ‘John’)
dc_body += """"user2" [URL="%s", url="%s", label="%s", shape="box", \
style="filled", fillcolor="yellow"];\n""" % (‘user2′, ‘user2′, ‘Steven’)
dc_body += """"user3" [URL="%s", url="%s", label="%s", shape="box", \
style="filled", fillcolor="yellow"];\n""" % (‘user3′, ‘user3′, ‘Paul’)
dc_body += """"user4" [URL="%s", url="%s", label="%s", shape="box", \
style="filled", fillcolor="yellow"];\n""" % (‘user4′, ‘user4′, ‘Thomas’)
# plays
dc_body += """"tennis" [URL="%s", url="%s", label="%s", shape="circle", \
style="filled", fillcolor="green"];\n""" % (‘tennis’, ‘tennis’, ‘Tennis’)
dc_body += """"basket" [URL="%s", url="%s", label="%s", shape="circle", \
style="filled", fillcolor="lightblue"];\n""" % (‘basket’, ‘basket’, ‘Basket’)
dc_body += """"soccer" [URL="%s", url="%s", label="%s", shape="circle", \
style="filled", fillcolor="hotpink"];\n""" % (‘soccer’, ‘soccer’, ‘Soccer’)
dc_body += """"karate" [URL="%s", url="%s", label="%s", shape="circle", \
s tyle="filled", fillcolor="white"];\n""" % (‘karate’, ‘Karate’, ‘Karate’)

# Fotter
dc_footer = "}"

# Join all parts
xdot_code = dc_head + dc_body + dc_footer

# Update widget with the new dot code
self.set_dotcode(xdot_code)

# callback called after user clicks in a node
def on_url_clicked(widget, keynode, event):
print ‘Key Node:’, keynode

if __name__ == ‘__main__’:
# create widget and display dot code
xdot = FooXDot(on_url_clicked)
xdot.display_dot()
xdot.show_all()

# gtk stuff
window = gtk.Window()
window.resize(800, 600)
window.connect(‘destroy’, gtk.main_quit)
window.show_all()

# Embed XDot widget
window.add(xdot)

# Get fun
gtk.main()

Resultado:

XDot widget in action

Código fuente de otro ejemplo: xdot.zipSimilar Posts:

Attached Files:

posted in GUI by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

Question about UTF-8

Wed, 18 Aug 2010 14:56:11 +0000

Sometimes, I’ve noticed that my application doesn’t deal very well with UTF-8 input/output. UTF-8 is mandatory but it’s a headache when you have to spend your time guessing what is wrong and how display the right characteres. I’ve found this function to force encode any text into UTF-8.

keydict = {
‘zwnj’: ”, ‘aring’: ‘\xc3\xa5′, ‘gt’: ‘>’, ‘yen’: ‘\xc2\xa5′,
‘ograve’: ‘\xc3\xb2′, ‘Chi’: ‘\xce\xa7′, ‘delta’: ‘\xce\xb4′,
‘rang’: ‘\xe3\x80\x89′, ‘sup’: ‘\xe2\x8a\x83′,
‘trade’: ‘\xe2\x84\xa2′, ‘ge’: ‘\xe2\x89\xa5′, ‘xi’: ‘\xce\xbe’,
‘upsih’: ‘\xcf\x92′, ‘nbsp’: ”, ‘Atilde’: ‘\xc3\x83′,
‘radic’: ‘\xe2\x88\x9a’, ‘otimes’: ‘\xe2\x8a\x97′,
‘aelig’: ‘\xc3\xa6′, ‘oelig’: ‘\xc5\x93′, ‘equiv’: ‘\xe2\x89\xa1′,
‘ni’: ‘\xe2\x88\x8b’, ‘le’: ‘\xe2\x89\xa4′, ‘auml’: ‘\xc3\xa4′,
‘cup’: ‘\xe2\x88\xaa’, ‘Epsilon’: ‘\xce\x95′, ‘otilde’: ‘\xc3\xb5′,
‘lt’: ‘< ‘, ‘Icirc’: ‘\xc3\x8e’, ‘shy’: ”, ‘Eacute’: ‘\xc3\x89′,
‘ndash’: ‘\xe2\x80\x93′, ‘yacute’: ‘\xc3\xbd’,
‘Prime’: ‘\xe2\x80\xb3′, ‘prime’: ‘\xe2\x80\xb2′, ‘psi’: ‘\xcf\x88′,
‘Kappa’: ‘\xce\x9a’, ‘rsaquo’: ‘\xe2\x80\xba’, ‘Tau’: ‘\xce\xa4′,
‘alefsym’: ‘\xe2\x84\xb5′, ‘ocirc’: ‘\xc3\xb4′, ‘lrm’: ”,
‘zwj’: ”, ‘cedil’: ‘\xc2\xb8′, ‘Ecirc’: ‘\xc3\x8a’,
‘not’: ‘\xc2\xac’, ‘Dagger’: ‘\xe2\x80\xa1′, ‘AElig’: ‘\xc3\x86′,
‘oslash’: ‘\xc3\xb8′, ‘acute’: ‘\xc2\xb4′, ‘lceil’: ‘\xe2\x8c\x88′,
‘iquest’: ‘\xc2\xbf’, ‘uacute’: ‘\xc3\xba’, ‘laquo’: ‘\xc2\xab’,
‘dArr’: ‘\xe2\x87\x93′, ‘rdquo’: ‘\xe2\x80\x9d’,
‘Ntilde’: ‘\xc3\x91′, ‘Igrave’: ‘\xc3\x8c’, ‘nu’: ‘\xce\xbd’,
‘lsquo’: ‘\xe2\x80\x98′, ‘lsaquo’: ‘\xe2\x80\xb9′,
‘sube’: ‘\xe2\x8a\x86′, ‘rarr’: ‘\xe2\x86\x92′,
‘sdot’: ‘\xe2\x8b\x85′, ‘supe’: ‘\xe2\x8a\x87′, ‘Yacute’: ‘\xc3\x9d’,
‘lfloor’: ‘\xe2\x8c\x8a’, ‘lArr’: ‘\xe2\x87\x90′, ‘Auml’: ‘\xc3\x84′,
‘asymp’: ‘\xe2\x89\x88′, ‘Otilde’: ‘\xc3\x95′, ‘szlig’: ‘\xc3\x9f’,
‘clubs’: ‘\xe2\x99\xa3′, ‘agrave’: ‘\xc3\xa0′, ‘Ocirc’: ‘\xc3\x94′,
‘Theta’: ‘\xce\x98′, ‘Pi’: ‘\xce\xa0′, ‘harr’: ‘\xe2\x86\x94′,
‘Scaron’: ‘\xc5\xa0′, ‘frac14′: ‘\xc2\xbc’, ‘egrave’: ‘\xc3\xa8′,
‘sub’: ‘\xe2\x8a\x82′, ‘iexcl’: ‘\xc2\xa1′, ‘frac12′: ‘\xc2\xbd’,
‘ordf’: ‘\xc2\xaa’, ‘rlm’: ”, ‘prop’: ‘\xe2\x88\x9d’,
‘circ’: ‘\xcb\x86′, ‘ntilde’: ‘\xc3\xb1′, ‘apos’: "’",
‘atilde’: ‘\xc3\xa3′, ‘THORN’: ‘\xc3\x9e’, ‘theta’: ‘\xce\xb8′,
‘prod’: ‘\xe2\x88\x8f’, ‘nsub’: ‘\xe2\x8a\x84′,
‘hArr’: ‘\xe2\x87\x94′, ‘rArr’: ‘\xe2\x87\x92′, ‘Oslash’: ‘\xc3\x98′,
‘omicron’: ‘\xce\xbf’, ‘infin’: ‘\xe2\x88\x9e’, ‘yuml’: ‘\xc3\xbf’,
‘Mu’: ‘\xce\x9c’, ‘Psi’: ‘\xce\xa8′, ‘thinsp’: ‘\xe2\x80\x89′,
‘ecirc’: ‘\xc3\xaa’, ‘bdquo’: ‘\xe2\x80\x9e’, ‘Sigma’: ‘\xce\xa3′,
‘kappa’: ‘\xce\xba’, ‘Aring’: ‘\xc3\x85′, ’tilde’: ‘\xcb\x9c’,
‘emsp’: ‘\xe2\x80\x83′, ‘mdash’: ‘\xe2\x80\x94′,
‘uarr’: ‘\xe2\x86\x91′, ‘permil’: ‘\xe2\x80\xb0′, ‘tau’: ‘\xcf\x84′,
‘Ugrave’: ‘\xc3\x99′, ‘fnof’: ‘\xc6\x92′, ‘Agrave’: ‘\xc3\x80′,
‘sup1′: ‘\xc2\xb9′, ‘forall’: ‘\xe2\x88\x80′, ‘eth’: ‘\xc3\xb0′,
‘rceil’: ‘\xe2\x8c\x89′, ‘iuml’: ‘\xc3\xaf’, ‘gamma’: ‘\xce\xb3′,
‘lambda’: ‘\xce\xbb’, ‘reg’: ‘\xc2\xae’, ‘Egrave’: ‘\xc3\x88′,
‘sup3′: ‘\xc2\xb3′, ‘dagger’: ‘\xe2\x80\xa0′, ‘divide’: ‘\xc3\xb7′,
‘Ouml’: ‘\xc3\x96′, ‘image’: ‘\xe2\x84\x91′, ‘hellip’: ‘\xe2\x80\xa6′,
‘igrave’: ‘\xc3\xac’, ‘Yuml’: ‘\xc5\xb8′, ‘Oacute’: ‘\xc3\x93′,
‘pound’: ‘\xc2\xa3′, ‘eacute’: ‘\xc3\xa9′, ‘frasl’: ‘\xe2\x81\x84′,
‘ETH’: ‘\xc3\x90′, ‘lowast’: ‘\xe2\x88\x97′, ‘Nu’: ‘\xce\x9d’,
‘plusmn’: ‘\xc2\xb1′, ‘chi’: ‘\xcf\x87′, ‘sup2′: ‘\xc2\xb2′,
‘frac34′: ‘\xc2\xbe’, ‘Aacute’: ‘\xc3\x81′, ‘cent’: ‘\xc2\xa2′,
‘oline’: ‘\xe2\x80\xbe’, ‘Beta’: ‘\xce\x92′, ‘perp’: ‘\xe2\x8a\xa5′,
‘Delta’: ‘\xce\x94′, ‘loz’: ‘\xe2\x97\x8a’, ‘pi’: ‘\xcf\x80′,
‘iota’: ‘\xce\xb9′, ‘empty’: ‘\xe2\x88\x85′, ‘euml’: ‘\xc3\xab’,
‘brvbar’: ‘\xc2\xa6′, ‘iacute’: ‘\xc3\xad’, ‘para’: ‘\xc2\xb6′,
‘ordm’: ‘\xc2\xba’, ‘epsilon’: ‘\xce\xb5′, ‘uuml’: ‘\xc3\xbc’,
‘OElig’: ‘\xc5\x92′, ‘there4′: ‘\xe2\x88\xb4′,
‘part’: ‘\xe2\x88\x82′, ‘icirc’: ‘\xc3\xae’, ‘bull’: ‘\xe2\x80\xa2′,
‘upsilon’: ‘\xcf\x85′, ‘copy’: ‘\xc2\xa9′, ‘Iuml’: ‘\xc3\x8f’,
‘Lambda’: ‘\xce\x9b’, ‘Xi’: ‘\xce\x9e’, ‘ensp’: ‘\xe2\x80\x82′,
‘ccedil’: ‘\xc3\xa7′, ‘Ucirc’: ‘\xc3\x9b’, ‘cap’: ‘\xe2\x88\xa9′,
‘Upsilon’: ‘\xce\xa5′, ‘mu’: ‘\xce\xbc’, ‘sigmaf’: ‘\xcf\x82′,
‘scaron’: ‘\xc5\xa1′, ‘nabla’: ‘\xe2\x88\x87′,
‘isin’: ‘\xe2\x88\x88′, ‘Zeta’: ‘\xce\x96′, ‘minus’: ‘\xe2\x88\x92′,
‘deg’: ‘\xc2\xb0′, ‘and’: ‘\xe2\x88\xa7′, ‘real’: ‘\xe2\x84\x9c’,
‘ang’: ‘\xe2\x88\xa0′, ‘curren’: ‘\xc2\xa4′, ‘int’: ‘\xe2\x88\xab’,
‘ucirc’: ‘\xc3\xbb’, ‘rfloor’: ‘\xe2\x8c\x8b’, ‘crarr’: ‘\xe2\x86\xb5′,
‘ugrave’: ‘\xc3\xb9′, ‘notin’: ‘\xe2\x88\x89′, ‘exist’: ‘\xe2\x88\x83′,
‘cong’: ‘\xe2\x89\x85′, ‘Uuml’: ‘\xc3\x9c’, ‘oplus’: ‘\xe2\x8a\x95′,
‘times’: ‘\xc3\x97′, ‘Acirc’: ‘\xc3\x82′, ‘piv’: ‘\xcf\x96′,
‘Euml’: ‘\xc3\x8b’, ‘Phi’: ‘\xce\xa6′, ‘raquo’: ‘ \xc2\xbb’,
‘Iacute’: ‘\xc3\x8d’, ‘quot’: ”, ‘Uacute’: ‘\xc3\x9a’,
‘Omicron’: ‘\xce\x9f’, ‘ne’: ‘\xe2\x89\xa0′, ‘Iota’: ‘\xce\x99′,
‘eta’: ‘\xce\xb7′, ‘sbquo’: ‘\xe2\x80\x9a’, ‘Rho’: ‘\xce\xa1′,
‘darr’: ‘\xe2\x86\x93′, ‘Alpha’: ‘\xce\x91′, ‘zeta’: ‘\xce\xb6′,
‘Omega’: ‘\xce\xa9′, ‘acirc’: ‘\xc3\xa2′, ‘sim’: ‘\xe2\x88\xbc’,
‘phi’: ‘\xcf\x86′, ‘diams’: ‘\xe2\x99\xa6′, ‘macr’: ‘\xc2\xaf’,
‘larr’: ‘\xe2\x86\x90′, ‘Ccedil’: ‘\xc3\x87′, ‘aacute’: ‘\xc3\xa1′,
‘uArr’: ‘\xe2\x87\x91′, ‘beta’: ‘\xce\xb2′, ‘Eta’: ‘\xce\x97′,
‘weierp’: ‘\xe2\x84\x98′, ‘rho’: ‘\xcf\x81′, ‘micro’: ‘\xc2\xb5′,
‘alpha’: ‘\xce\xb1′, ‘omega’: ‘\xcf\x89′, ‘middot’: ‘\xc2\xb7′,
‘Gamma’: ‘\xce\x93′, ‘euro’: ‘\xe2\x82\xac’, ‘lang’: ‘\xe3\x80\x88′,
‘spades’: ‘\xe2\x99\xa0′, ‘amp’: ‘and’, ‘rsquo’: ‘\xe2\x80\x99′,
‘uml’: ‘\xc2\xa8′, ‘thorn’: ‘\xc3\xbe’, ‘ouml’: ‘\xc3\xb6′,
‘thetasym’: ‘\xcf\x91′, ‘or’: ‘\xe2\x88\xa8′, ‘Ograve’: ‘\xc3\x92′,
‘sect’: ‘\xc2\xa7′, ‘ldquo’: ‘\xe2\x80\x9c’,
‘hearts’: ‘\xe2\x99\xa5′, ‘sigma’: ‘\xcf\x83′,‘sum’: ‘\xe2\x88\x91′,
‘oacute’: ‘\xc3\xb3′
}

def decode_unicode_references(data):
try:
text = data
pos1 = text.find(‘&’)
pos2 = text.find(‘;’)
if (((pos1 != -1) or (pos2 != -1)) and (pos2 – pos1 < 15)):
entity = text[pos1+1: pos2]
a = text[pos1:pos2+1]
return text.replace(a, keydict[entity])
else:
return data
except:
return data

def str2utf8(data):
data = decode_unicode_references(data)
guess_list=[‘utf-8′,‘iso8859-1′,‘iso8859-2′, \
‘iso8859-15′,‘ascii’]
encoding = ‘iso8859-1′
text = None
string = None
for best_enc in guess_list:
try:
encoding = best_enc
text = data.decode(encoding).encode(‘utf-8′)
break
except Exception, error:
#print "ERROR:", error
pass

return text

How do you deal with UTF-8, python and gtk? Am I doing right? Do you know a better way?

Similar Posts:

posted in Python by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

Reports with Cheetah templates

Mon, 16 Aug 2010 09:32:37 +0000

I’ve found Cheetah template engine very useful to build custom reports. With a very small set of syntactic rules you can mix html and python to generate dynamic content.

This is a small example extracted from my project:

filename default.html:

#import time
#from vazaarlib.i18n import _
#from vazaarlib.utils import fuzzy_date, str2utf8, get_resource_contents
#from vazaarlib.utils import get_beautiful_rtype
#from vazaarlib.namespaces import *
#from vazaarlib.constants import GPATH, LPATH

#set $user_owner_pim_uri = $self.gui.app.cfgmgr.get_value(‘PIMO’, ‘PIM’)
#set $user_owner_uri = $self.gui.ask.get_owner($user_owner_pim_uri)
#set $user_owner_email_uri = $self.gui.ask.get_owner_email($user_owner_uri)
#set $uemail = $self.gui.app.vstore.get_property($user_owner_email_uri)
#set $uowner = $self.gui.app.vstore.get_property($user_owner_uri)
#set $totres = $self.gui.ask.get_total_resources()
#set $typefreq = $self.gui.ask.get_qname_frequency($RDF.type)
#set $medate = time.strftime("%A, %d %b %Y", $time.localtime(time.time()))
#set $metime = time.strftime("%H:%M", $time.localtime(time.time()))
#set $css = $open($GPATH['CSS'] + ‘/default/style.css’, ‘r’).read()

< !DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">

Vazaar
$css
–>

$medate @ $metime

$_("Hi") $uowner, your Knowledge Database has $totres resources

#for $rtype in $typefreq:
#set $title = get_beautiful_rtype($rtype)
#set $freq = $typefreq[$rtype]

#end for

"font-size: 10pt;" cellspacing="3px" cellpadding="3px" width="100%">

id="row">

"border-bottom: 1px solid gray;" align="left">$title	"border-bottom: 0px dotted gray;" align="right">$freq
"border-top: 0px solid black; color: black;" align="right">$_("Total resources: ")	"border-top: 1px solid black; color: black;" align="right">$totres

filename reports.py:

#!/usr/bin/python
# -*- coding: utf-8 -*-
# File: reports.py
# Author: Tomás Vírseda
# License: GPL v3
# Description: Report builder for Vazaar

[…]
from Cheetah.Template import Template

class Report():
def __init__(self, gui):
…
…

def build_starthere_report(self, params):
template = GPATH[‘TEMPLATES’] + ‘/default.html’
report = Template(file=template)
report.css_content = self.css_content
report.author = ‘Tomás Vírseda García’
report.date = ’14/12/2010′
report.params = params

return str(report)
[…]

Check the code at vazaarlib/ui/reports.py in the source project repository.Similar Posts:

posted in Reports by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

Searching strings

Wed, 21 Jul 2010 10:53:51 +0000

Well, another thing I learnt today: how to search a given string in a RDF graph but matching only Literals:

def by_term(self, search_term):
query = """
SELECT DISTINCT ?rid
WHERE {
?rid ?predicate ?object .
FILTER isLiteral(?object) .
FILTER regex(?object, "%s", "i")
}
ORDER BY ?object
""" % Literal(search_term)
result = run_query(query, self.graph)

return result.selected

The most important here is the operator isLiteral used in the filters.Similar Posts:

posted in SPARQL Queries by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

distutils implemented

Sat, 12 Sep 2009 00:48:39 +0000

Finally, I got the way to implement distutils into Vazaar. It lacks support for i18n/gettext but it is in the way. I’ve released a new tarball. You can launch it as developer inside your home with the command ./launch

But if you prefer a wide-system install, you must use the following command:

sudo python2.5 setup.py install

You can find an entry under Applications -> Office -> Vazaar

python 2.6 also works but it throws warnings.

Please, note that you can NOT combine different versions of python because database sqlite formats differs each from other.Similar Posts:

posted in Python by Tomás Vírseda Leave A Comment
©2010 Vazaar Project Blog. All Rights Reserved.

Dynamic SPARQL queries and Datatypes

Wed, 26 Aug 2009 09:24:44 +0000

After some headaches I’ve found how to write dynamic SPARQL queries by passing parameters. For example, to query graph about resources collected last hour this code could be a first approach:

#!/usr/bin/env python
# Dynamic query (example code)

from datetime import datetime
import rdflib
from namespaces import XSD

def last_day():
""" get date """
now = datetime.now()
date = now – timedelta(hours=24)

return date

def get_date_filtered_query():
""" build query """
period = last_day()
mydate = Literal(period, datatype=URIRef(XSD))

query = """
PREFIX rdf:
PREFIX nie:
PREFIX nfo:
PREFIX nao:
PREFIX pimo:
PREFIX xsd:
SELECT ?id
WHERE
{
?id rdf:type ?type .
?id nao:created ?date .
FILTER (?type != pimo:Collection) .
FILTER (xsd:dateTime(?date) >= xsd:dateTime("%s")) .
}
ORDER BY DESC(?date)
""" % mydate

return query

# query graph
query = get_date_filtered_query()
result = graph.query(query)

for id in result.selected:
print id

For more information is worth to take a look W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes. A self explained list of XML datatypes:

xsd:anyURI – URI (Uniform Resource Identifier)
xsd:base64Binary – Binary content coded as “base64″
xsd:boolean – Boolean (true or false)
xsd:byte – Signed value of 8 bits
xsd:date – Gregorian calendar date
xsd:dateTime – Instant of time (Gregorian calendar)
xsd:decimal – Decimal numbers
xsd:double – IEEE 64-bit floating-point
xsd:duration – Time durations
xsd:ENTITIES – Whitespace-separated list of unparsed entity references
xsd:ENTITY – Reference to an unparsed entity
xsd:float – IEEE 32-bit floating-point
xsd:gDay – Recurring period of time: monthly day
xsd:gMonth – Recurring period of time: yearly month
xsd:gMonthDay – Recurring period of time: yearly day
xsd:gYear – Period of one year
xsd:gYearMonth – Period of one month
xsd:hexBinary – Binary contents coded in hexadecimal
xsd:ID – Definition of unique identifiers
xsd:IDREF – Definition of references to unique identifiers
xsd:IDREFS – Definition of lists of references to unique identifiers
xsd:int – 32-bit signed integers
xsd:integer – Signed integers of arbitrary length
xsd:language – RFC 1766 language codes
xsd:long – 64-bit signed integers
xsd:Name – XML 1.O name
xsd:NCName – Unqualified names
xsd:negativeInteger – Strictly negative integers of arbitrary length
xsd:NMTOKEN – XML 1.0 name token (NMTOKEN)
xsd:NMTOKENS – List of XML 1.0 name tokens (NMTOKEN)
xsd:nonNegativeInteger – Integers of arbitrary length positive or equal to zero
xsd:nonPositiveInteger – Integers of arbitrary length negative or equal to zero
xsd:normalizedString – Whitespace-replaced strings
xsd:NOTATION – Emulation of the XML 1.0 feature
xsd:positiveInteger – Strictly positive integers of arbitrary length
xsd:QName – Namespaces in XML-qualified names
xsd:short – 32-bit signed integers
xsd:string – Any string
xsd:time – Point in time recurring each day
xsd:token – Whitespace-replaced and collapsed strings
xsd:unsignedByte – Unsigned value of 8 bits
xsd:unsignedInt – Unsigned integer of 32 bits
xsd:unsignedLong – Unsigned integer of 64 bits
xsd:unsignedShort – Unsigned integer of 16 bits

Similar Posts:

Querying remote ontologies from integrated SPARQL editor

Sun, 02 Aug 2009 16:28:44 +0000

Checking others parts of the Vazaar GUI, I’ve realized that I can query remote ontologies and vocabularies with a simple SPARQL sentence. In the screenshot you can see how I query for the comments of all NAO entities.

Just for the record, it would be nice to save user queries and integrate them within Vazaar. And, of course, to export result data to a file (eg.: in a CSV file) ready to import to a any spreadsheet program like OpenOffice Calc.Similar Posts:

Attached Files:

Querying remote ontologies
By using the integrated SPARQL editor it is possible to query remote ontologies and vocabularies

GUI improvements

Sun, 02 Aug 2009 14:02:17 +0000

After a long weekend coding and trying new design ideas finally I think I got a nice GUI. At least for the StartHere module:

There are still too much glitches in the GUI but I’m trying to fix them as I go. But for now, I feel satisfied. Queries against the model are working as expected.

After clicking in any stats button you get a nice visual for your resources. In next commits, you will be able of choosing one the two kinds of views: ListViews and IconViews. By default, the IconView is preferred.

More ideas are coming like content filtering, properties tagcloud, timeline widget, etc…

At this moment, you can only add new resources by copy&paste operations. An improved dialog for this operation is in the way. Resources can be selected for be deleted. If the selected resource is a collection, all resources which belong to this collection will be deleted. There is no actions popup menu yet but it’s planned.

I’m thinking if a Trash resource is worth. NEPOMUK brings especial classes for this cuestion. I need check twice these. Anyway, it wouldn’t be very difficult.

One of the most important things now is to code common operation for resources: tagging, annotations, relations between them and how to display their content. For example, display text or an image is easy to do but it is not very easy to display websites content. There is not a good browser widget ready to use in PyGTK yet. I put my hope in webkit bindings for Python.Similar Posts:

Attached Files:

Screenshot for Vazaar 0.4b

Extrayendo metadatos: libextractor

Sun, 02 Dec 2007 19:41:15 +0000

Para que Vazaar sea realmente útil debe extraer un alto porcentaje de metadatos que casi haga innecesaria la interacción del usuario. Hay una entrada en el wiki dedicada al análisis de utilidades y/o bibliotecas útiles para el proyecto. Ahora estoy probando con libextractor.

libextractor es una libreria cuyo fin es extraer metadatos de archivos de cualquier tipo. Se diseñó para utilizar otras liberias para ayudar en el proceso concreto de extraccion de metadatos, y tambien para que se pudiera ampliar facilmente su función con solo enlazar con extractores externos para tipos de archivo adicionales. libextractor es parte del proyecto GNU.

Lo interesante de esta herramienta es la cantidad de metadatos que es capaz de extraer. Este es un listado de los tags soportados por libextractor en mi sistema:

from extractor import Extractor
extractor = Extractor()
extractor.keywordTypes()
(‘unknown’, ‘filename’, ‘mimetype’, ‘title’, ‘author’, ‘artist’, ‘description’, ‘comment’, ‘date’, ‘publisher’,‘language’,
‘album’, ‘genre’, ‘location’, ‘version’, ‘organization’, ‘copyright’, ‘subject’, ‘keywords’, ‘contributor’,
‘resource-type’, ‘format’, ‘resource-identifier’, ‘source’, ‘relation’, ‘coverage’, ‘software’, ‘disclaimer’, ‘warning’,
‘translated’, ‘creation date’, ‘modification date’, ‘creator’, ‘producer’, ‘page count’, ‘page orientation’,
‘paper size’, ‘used fonts’, ‘page order’, ‘created for’, ‘magnification’, ‘release’, ‘group’, ‘size’, ‘summary’,
‘packager’, ‘vendor’, ‘license’, ‘distribution’, ‘build-host’, ‘os’, ‘dependency’, ‘MD4′, ‘MD5′, ‘SHA-0′, ‘SHA-1′,
‘RipeMD160′, ‘resolution’, ‘category’, ‘book title’, ‘priority’, ‘conflicts’, ‘replaces’, ‘provides’, ‘conductor’, ‘interpreter’,
‘owner’, ‘lyrics’, ‘media type’, ‘contact’, ‘binary thumbnail data’, ‘publication date’, ‘camera make’, ‘camera model’,
‘exposure’, ‘aperture’, ‘exposure bias’, ‘flash’, ‘flash bias’, ‘focal length’, ‘focal length (35mm equivalent)’, ‘iso speed’,
‘exposure mode’, ‘metering mode’, ‘macro mode’, ‘image quality’, ‘white balance’, ‘orientation’)

A pesar de la gran cantidad de metadatos que puede extraer no facilita manera alguna de saber con que tipo de metadato estás jugando. Según los creadores de esta librería, la mayor parte de los metadatos extraídos están basados en el vocabulario de Dublin Core. Sin embargo, otros metadatos han sido añadidos ante la necesidad de ciertos tipos de formato. Es decir, extrae un conjunto de parejas tipo clave/valor sin orden ni concierto dejando a la imaginación de los programadores la interpretación de los mismos.

En vazaar, MDX es el módulo encargado de la extracción de metadatos. Dado que libextractor está en la mayor parte de los sistemas y existen bindings para python me ha parecido razonable incluirlo. El mayor problema que estoy encontrando para implementarlo es la cantidad de tiempo que gasto buscando que significa cada cosa. Para ello estoy mapeando cada keyword de libextractor con su correspondiente elemento en Dublin Core. También tengo en cuenta los elementos cualificados:

# keyword['LIBEXTRACTOR KEYWORD-TYPE'] = (‘DC-ELEMENT’, ‘DC-TERM’)
keyword[‘album’] = (‘type’, ‘collection’)
keyword[‘aperture’] = (”, ”)
keyword[‘artist’] = (‘creator’, ”)
keyword[‘author’] = (‘creator’, ”)
keyword[‘binary thumbnail data’] = (”, ”)
keyword[‘book title’] = (‘title’, ”)
keyword[‘build-host’] = (”, ”)
…

Ejemplo del uso de MDX:

kaskaras@laptop:~/share/wc/vazaar/trunk/devel/swx$ python mdx.py screenshot.png
Adding item: screenshot.png
http://purl.org/dc/elements/1.1/description -> Created with GIMP
http://purl.org/dc/terms/modified -> 2007-11-01 22:04:54
http://purl.org/dc/elements/1.1/format -> image/png

Updated 19/XII/2009:
extract-metadata blueprint @ launchpad.netSimilar Posts:

Initial import

Thu, 01 Nov 2007 21:41:56 +0000

Por fin me he decidido a hacer el Initial import de lo que tengo en este momento. Tan solo una prueba de concepto para aprender a manejarme con python, rdf y pygtk.

Se trata de un explorador/visor de ficheros (si se lo puede llamar así) ala web semántica. Mapea un directorio dado en un grafo y muestra los triples, metadatos encontrados* y una imagen del grafo del directorio actual.

Este módulo (al que he llamado swx) se encargará en un futuro de añadir ficheros y directorios al repositorio central así como de proveer de metadatos la base de datos de conocimiento de Vazaar.

El código:

kaskaras@laptop:~/share/wc$ svn co http://vazaar-project.org/svn/trunk/
A    trunk/devel
A    trunk/devel/swx
A    trunk/devel/swx/vazaar.glade
A    trunk/devel/swx/RdfDir.py
A    trunk/devel/swx/file.xpm
A    trunk/devel/swx/namespaces.py
A    trunk/devel/swx/swx.py
A    trunk/devel/swx/folder.xpm
A    trunk/frozen
A    trunk/play
Revisión obtenida: 1

Pantallazo:

* letra pequeña: obtiene pocos metadatos. Falta desarrollar el módulo que se encargue de esta parte

Actualización: instala las utilidades de Redland (no he conseguido hacer funcionar bien pydot) para ver convertir RDF a DOT y poder ver la imagen del grafo.Similar Posts: