I downloaded the MeSH ascii format after using this module to create the hierarchical taxonomy and used the ENTRY field to add synonyms to the taxonomy via the attached python script. Just download the d2011.bin file and edit the python script with your database info and it adds about 120k synonyms to your taxonomy.

Comments

groovypower’s picture

code


import MySQLdb, string

db=MySQLdb.connect(host="host",user="user",passwd="password",db="database")
cursor = db.cursor()

def getAll(cursor, table, item, where):
        query = "SELECT %s FROM %s WHERE `name` LIKE '%s'" % (item,table,where)
        cursor.execute(query)
        r = cursor.fetchall()
        ret = []
        for first in r: ret.append((first))
        return ret

def undupe(word):
	rules = {'ss' : False, 
        'os' : 'o', 
        'ies' : 'y', 
        'xes' : 'x', 
        'oes' : 'o', 
        'ies' : 'y', 
        'ves' : 'f', 
        's' : ''}
	for key in rules.keys():
		if word.endswith(key) == False:
			continue
		if rules[key] == False:
			return word
		l = 0-len(key)
		return word[0:l] + rules[key]
	return word
count =0
tax = ""

tid = ""
des = ""
syn = []
current = []
f = open("d2011.bin")
for line in  f:
	if "NEWRECORD" in line:
		count = count + 1
		tax = ""
		tid = ""
		des = ""
		syn = []
		current = []
		nextR = True
		nextR = True
	elif "MH =" in line and nextR:
		nextR = False
		tax = line.split('=')[1].strip()
		try:
			nf = getAll(cursor, "`term_data`", "`tid`, `description`", tax)
			for n in nf:
				current.append(n)
		except:
			continue
	elif "MN =" in line and tax != "":
		des = line.split('=')[1].strip()
		for c in current:
			if c is list:
				for t,d in c:
					if d in des:
						tid = t
					break	
			else:
				tid = c[0]
			if tid != "":
				break

	elif "ENTRY =" in line and tax != "":
		temp = line.split('=')[1].strip()
		if "|" not in temp:
			syn.append(temp)
	
	elif "UI =" in line and tid != "":
		for s in syn:
			s = string.replace(s, "\'","\\\'").strip()
			ins = "INSERT INTO `term_synonym` (`tid`, `name`) VALUES ('%d',  '%s')" % (tid, s) 
			cursor.execute(ins)
			s2 = undupe(s)
			if s2 != s:
				ins = "INSERT INTO `term_synonym` (`tid`, `name`) VALUES ('%d',  '%s')" % (tid, s2) 
				cursor.execute(ins)
		print count
f.close()
erikwebb’s picture

Priority: Normal » Major

I don't grok Python, what keys are you using to determine synonyms in the ASCII format. After taking a second look at ASCII, maybe that's a better option for a format to import. Any thoughts?

I'd like to have an assortment of importers, likely to help solve the issue of term updates.

erikwebb’s picture

Status: Active » Postponed (maintainer needs more info)
erikwebb’s picture

Status: Postponed (maintainer needs more info) » Closed (won't fix)