This is a class developed for Shuai Tang's neural network.
It will load and parse data from a hicopy object and pass it to the neural network
index/id of image and cat(hoi) starts from 1 instead of 0
index/ids must be integer and must be within the total number
Default location of the JSON file:'C:\\Users\\du_xi\\Dropbox\\Research Shuai Tang\\list.json'
Default location of the images:'C:\\Users\\du_xi\\.spyder-py3\\hico_20160224_det'
__getitem__ fucntion has been overrided. hicopyLoader[i] will return information of the ith image
The information returned by hicoipyLoader[i]: a dictionary 'inputs' that contains 3 sub-dictionaries - 'data', 'im_info' and 'labels'
'data' contains 'im', 'boxes' and 'box_cls':
'im': image in np array format (please see readImgs() in hicopy)
'boxes': an array of all bounding boxes in the given image
'box_cls': one-hot of all objects in the given image
'im_info' contains 'im_id', 'im_hw', 'ids' and 'person_ids':
'im_id': integer, index of the image
'im_hw': list with two elements, height and width, respectively
'ids': a list with the length of the number of objects in an image. The list starts from 0, increments 1 and ends at len(box_cls)-1
E.g. for an image with 6 objects, 'ids' should be [0,1,2,3,4,5]
'person_ids': a list of index in 'ids' that are 'human'
E.g. if object 0 and 4 are humen in the above example, 'person_ids' should be [0,4]
'labels' contains 'labels_prop' and 'labels_cls'
'labels_prop': one-hot format of the interaction between humen and objects. np array with dimension of (len(person_ids),len(ids))
labels_prop[i,j]=1 if person_ids[i] has interaction with ids[j]
=0 otherwise (human does not have any interactions with himself/herself)
'labels_cls': one-hot format of what kind of interaction between humen and objects.
labels_cls[i,j,k]=1 if person_ids[i] has interaction k with ids[j]
=0 otherwise
Functions:
non_max_suppression_fast: suppress given boxes for given threshhold
it will return a list of box index showing which box will be replaced.
e.g. if boxes 0,1,2,3,4 are the original boxes and box 0 can be represented by 2 and box 4 can be represented by box 1,
the function will return [2,1,2,3,1]
borrowed from https://www.pyimagesearch.com/2015/02/16/faster-non-maximum-suppression-python/
onehot: turns list into one-hot format
uniqueList: uniqify given list
borrowed from https://www.peterbe.com/plog/uniqifiers-benchmark
hicopyLoader Class:
-hicopyLoader: It will load and parse data from a hicopy object and pass it to the neural network
-non_maximum_suppression_filter: a filter for the program to do non maximum suppression with given threshhold
-get_nms_index_human: list, result of non-maximum suppression of human bounding boxes
-get_nms_index_all: list, result of non-maximum suppression of all bounding boxes
-__getitem__: overrides the default getitem function and returns dictionary 'inputs'
-getData: returns dictionary 'data'
-getBoxes: returns np array 'boxes'
-getBoxesHuman: returns a np array that contains all human bounding boxes
-getBoxesObject: returns a np array that contains all object bounding boxes
-getBoxcls: returns one-hot 'box_cls'
-getImInfo: returns dictionary 'im_info'
-getIm_hw: returns list 'im_hw'
-getIds: returns list 'ids'
-getPerson_ids: returns list 'person_ids'
-getLabels: returns dictionary 'labels'
-getLabels_prop: returns one-hot 'labels_prop'
-getLabels_cls: returns one-hot 'labels_cls'
-getNumOfNoInteraction: returns an integer of the total number of 'no_interactions' in the image
-getObjectsOfGivenPerson: given a certain person in a certain image, the function returns the indices (in 'ids') of which objects are related to the person
-getVerbsOfGivenPerson: given a certain person in a certain image, the function returns the indices (in 'ids')of which actions are related to the person
-getObjectTypeOfGivenPerson: given a certain person in a certain image, the function returns the ids of the objects that are related to the person
-visualize: visualizes a given image for each hoi
-visualize_box_conn_one: a helper function of visualize()
"""
defonehot(dataList,totalState):
"""
inputs:
dataList: a list of data
totalState: a list of all possible states
outputs:
a np array of one-hot
"""
onehotlist=list()
fori,objninenumerate(dataList):
temp=np.zeros(totalState)
temp[objn-1]=1
onehotlist.append(temp)
returnnp.asarray(onehotlist)
defnon_max_suppression_fast(boxes,overlapThresh):
"""
inputs:
boxes: a list of bounding boxes
overlapThresh: the threshold (between 0 and 1)
outputs:
replace: a list of which box should be replaced by which box
"""
# if there are no boxes, return an empty list
iflen(boxes)==0:
return[]
# if the bounding boxes integers, convert them to floats --
# this is important since we'll be doing a bunch of divisions
ifboxes.dtype.kind=="i":
boxes=boxes.astype("float")
# initialize the list of picked indexes
pick=[]
# grab the coordinates of the bounding boxes
x1=boxes[:,0]
y1=boxes[:,1]
x2=boxes[:,2]
y2=boxes[:,3]
# compute the area of the bounding boxes and sort the bounding
# boxes by the bottom-right y-coordinate of the bounding box
area=(x2-x1+1)*(y2-y1+1)
idxs=np.argsort(y2)
replace=idxs*0
# keep looping while some indexes still remain in the indexes
# list
whilelen(idxs)>0:
# grab the last index in the indexes list and add the
# index value to the list of picked indexes
last=len(idxs)-1
i=idxs[last]
pick.append(i)
# find the largest (x, y) coordinates for the start of
# the bounding box and the smallest (x, y) coordinates
# for the end of the bounding box
xx1=np.maximum(x1[i],x1[idxs[:last]])
yy1=np.maximum(y1[i],y1[idxs[:last]])
xx2=np.minimum(x2[i],x2[idxs[:last]])
yy2=np.minimum(y2[i],y2[idxs[:last]])
# compute the width and height of the bounding box
